Expected Behavior
When the DynamoDB online store is initialized via initialize(), connections to DynamoDB should be pre-established so that the first get_online_features() call does not incur cold-start latency.
Current Behavior
The initialize() method creates the aiobotocore client but does not establish actual TCP connections to DynamoDB. The first BatchGetItem request incurs additional latency (~10-20ms) for connection establishment, TLS handshake, and authentication.
# Current implementation - client created but no connections established
async def initialize(self, config: RepoConfig):
online_config = config.online_store
await self._get_aiodynamodb_client(...) # Creates client object only
Steps to reproduce
import time
from feast import FeatureStore
fs = FeatureStore(repo_path=".")
# First request - includes connection setup overhead
start = time.time()
fs.get_online_features(features=["fv:f1"], entity_rows=[{"id": "1"}])
first_request = time.time() - start # ~80ms
# Second request - connections already warm
start = time.time()
fs.get_online_features(features=["fv:f1"], entity_rows=[{"id": "1"}])
second_request = time.time() - start # ~60ms
print(f"First: {first_request*1000:.0f}ms, Second: {second_request*1000:.0f}ms")
# Output: First: 80ms, Second: 60ms (20ms cold-start overhead)
Specifications
- Version: 0.60.0 and latest master
- Platform: Any
- Subsystem: feast.infra.online_stores.dynamodb
Possible Solution
Add a lightweight DynamoDB API call in initialize() to warm up connections:
async def initialize(self, config: RepoConfig):
online_config = config.online_store
client = await self._get_aiodynamodb_client(
online_config.region,
online_config.max_pool_connections,
# ... other params
)
# Warm up connection pool with a lightweight call
if online_config.warmup_connections: # New config option
try:
await client.describe_limits()
except Exception:
pass # Ignore errors, just warming up connections
Optionally gated behind a new config parameter warmup_connections: bool = False to maintain backward compatibility.
Expected Behavior
When the DynamoDB online store is initialized via initialize(), connections to DynamoDB should be pre-established so that the first get_online_features() call does not incur cold-start latency.
Current Behavior
The initialize() method creates the aiobotocore client but does not establish actual TCP connections to DynamoDB. The first BatchGetItem request incurs additional latency (~10-20ms) for connection establishment, TLS handshake, and authentication.
# Current implementation - client created but no connections established async def initialize(self, config: RepoConfig): online_config = config.online_store await self._get_aiodynamodb_client(...) # Creates client object onlySteps to reproduce
import time from feast import FeatureStore fs = FeatureStore(repo_path=".") # First request - includes connection setup overhead start = time.time() fs.get_online_features(features=["fv:f1"], entity_rows=[{"id": "1"}]) first_request = time.time() - start # ~80ms # Second request - connections already warm start = time.time() fs.get_online_features(features=["fv:f1"], entity_rows=[{"id": "1"}]) second_request = time.time() - start # ~60ms print(f"First: {first_request*1000:.0f}ms, Second: {second_request*1000:.0f}ms") # Output: First: 80ms, Second: 60ms (20ms cold-start overhead)Specifications
Possible Solution
Add a lightweight DynamoDB API call in initialize() to warm up connections:
async def initialize(self, config: RepoConfig): online_config = config.online_store client = await self._get_aiodynamodb_client( online_config.region, online_config.max_pool_connections, # ... other params ) # Warm up connection pool with a lightweight call if online_config.warmup_connections: # New config option try: await client.describe_limits() except Exception: pass # Ignore errors, just warming up connectionsOptionally gated behind a new config parameter warmup_connections: bool = False to maintain backward compatibility.