Kinesis Data Streams

Kinesis Data Streams is for use cases that require custom processing, choice of stream processing frameworks, and sub-second processing latency.

Each shard can support up to 1,000 PUT records per second. However, you can increase the number of shards limitlessly. One shard provides a capacity of 1 MB/sec data input and 2 MB/sec data output.

Amazon Kinesis Data Streams synchronously replicates data across three Availability Zones, providing high availability and data durability.

You pay per shard hour and per PUT payload unit. Optionally, there are fees associated with extended data retention and enhanced fan-out.

Kinesis Data Firehose

Kinesis Data Firehose is for use cases that require zero administration, the ability to use existing analytics tools on Amazon S3, Amazon Redshift, and Amazon Elasticsearch (Amazon ES), or if you require a data latency of 60 seconds or higher.

Kinesis Data Firehose will automatically scale to match the throughput of your data, without any manual intervention or developer overhead.

Amazon Kinesis Data Firehose synchronously replicates data across three Availability Zones, providing high availability and data durability.

You pay for the volume of data you ingest using the service and for any data format conversions.

When you enable Kinesis Data Firehose data transformation, Kinesis Data Firehose buffers incoming data and invokes the specified AWS Lambda function with each buffered batch asynchronously. The transformed data is sent from Lambda to Kinesis Data Firehose for buffering and then delivered to the destination. You can also choose to enable source record backup, which backs up all untransformed records to your S3 bucket concurrently while delivering transformed records to the destination.

Kinesis Data Firehose can also batch, compress, and encrypt the data before loading it, minimizing the amount of storage used at the destination and increasing security.