Redis Clusters
Redis clusters are fully managed (powered by Amazon ElastiCache for Redis). They are easy to set up, operate and scale. You don't have to worry about capacity scaling, hardware & VM provisioning, cluster setup, patching or backups.
Redis is a fast, NoSQL in-memory data store that can provide sub-millisecond latency.
Amazon ElastiCache redis clusters are built on open-source Redis and compatible with the Redis APIs. They work with Redis clients and use the open Redis data format to store your data.
When to use
Redis is used in cases when the performance is important. Example use-cases include caching, chat apps, messaging, queues, real-time analytics, etc.
Advantages
- Performance - Redis is an in-memory data store and can deliver sub-millisecond response times. In comparison to disk-based databases (where most operations need a roundtrip to a disk), in-memory data stores can be an order of magnitude faster.
- Fully managed - You don't have to worry about capacity scaling, hardware & VM provisioning, cluster setup, sharding process, patching, backups, etc.
- High availability - Redis supports both clustered and non-clustered modes and provides high availability via support for automatic failovers with minimal impact.
Disdvantages
- Data persistence - Redis offers 2 options for persistency: regular snapshotting and append-only files. None of them is as secure as a real transactional server providing undo/redo logging, point-in-time recovery, etc.
- Hard to store large amounts of data - All your data must fit in memory.
- Requires topology-aware clients - Clients connecting to the Redis cluster should be aware of the cluster topology. This requires more configuration on the client.
Basic usage
- Following example shows basic usage of
redis-cluster
resource. Only required properties are nodeInstanceSize and defaultUserPassword. - The example shows a single node
redis-cluster
.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')myFunction:type: functionproperties:packageConfig:filePath: path/to/my/lambda.tsenvironment:- name: REDIS_HOSTvalue: $Param('redisCluster', 'RedisCluster::PrimaryEndPoint.Address')- name: REDIS_PORTvalue: $Param('redisCluster', 'RedisCluster::PrimaryEndPoint.Port')- name: REDIS_PASSWORDvalue: $This().resources.myRedisCluster.properties.defaultUserPassword
Lambda function connected to a single-node redis cluster
import { Cluster } from 'ioredis';const redisClient = new Redis({host: process.env.REDIS_HOST,port: Number(process.env.REDIS_PORT),password: process.env.REDIS_PASSWORD});const handler = async (event, context) => {await redisClient.set('currentTime', `${Date.now()}`);const value = await redisClient.get('currentTime');return { result: value };};export default handler;
Lambda function storing and reading data from a single-node redis cluster
Node instance size
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')
Default user password
- Redis clusters are password protected. To communicate with the cluster, you must configure a password for the default user.
- Communication with the cluster is encrypted in-transit.
- You should not input the password directly. The recommended way is using a
.env
file together with File directive or even better, using secrets. - Password constraints:
- Must be only printable ASCII characters.
- Must be at least 16 characters and no more than 128 characters in length.
- Cannot contain any of the following characters: '/', '"', or '@'.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')
Cluster topology
- Cluster topology is based on the amount of replica nodes and the amount of shards.
- Performance and high-availability of the cluster are dependent on the topology.
Single node (non-sharded) cluster
- Single node, non-sharded cluster has only one node. It's not highly available.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')
Multi node (non-sharded) cluster
- Adding replica nodes (read replicas) to a cluster has 2 benefits:
- increases the read throughput
- increases availability. Replica node can become the primary node in case of primary node failure.
- Load balancing between replicas is automatically handeled by AWS.
- When there are multiple shards, this number specifies the number of replicas for each shard cluster.
Changing numReplicaNodes
when sharding is enabled is not possible and will result in error.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')numReplicaNodes: 2
Logging
- When logging is enabled, slow-log will be sent to a pre-created Cloudwatch log group.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')logging:enabled: true
Enables sending slow-log to a Cloudwatch log group.
Type: boolean
Configures the log format
Type: string ENUM
Possible values: jsontext
Amount of days the logs will be retained in the log group
Type: number ENUM
Possible values: 13571430609012015018036540054573118273653
Accessibility
- You can configure which resources and hosts can access your cluster.
- Redis clusters do not support public IP addresses. Therefore, only 2 access modes are supported.
Configures the accessibility mode for this cluster
Type: string ENUM
Possible values: scoping-workloads-in-vpcvpc
The following modes are supported:
- vpc - The cluster can be accessed only from resources within your VPC. This
means any function (provided it has
joinDefaultVpc
set to true), batch job or container workload within your stack can access the cluster. - scoping-workloads-in-vpc - similar to vpc mode, but even more restrictive. In addition to being in the same VPC, the resources and hosts
accessing your redis cluster must also have sufficient IAM permissions (for funtions, batch jobs and container workloads, these permissions
can be granted using
allowsAccessTo
oriamRoleStatements
in their configuration).
Redis clusters do not support public IP addresses. Therefore, you cannot connect to them from your own computer. To avoid this shortcoming, you can use a bastion server. Native support for bastion servers in Stacktape will be available soon.
To learn more about VPCs, refer to VPC Docs.
VPC mode
- The cluster can be accessed only from resources within the default VPC.
- Any function (provided it has
joinDefaultVpc
set to true), batch job or container workload within your stack can access the cluster. - Additionally, IP addresses configured in
whitelistedIps
can also access the database (even from the internet). To disable this behavior, and enforce database isolation ONLY to the VPC, you can set theforceDisablePublicIp
property.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')accessibility:accessibilityMode: vpc
Scoping workloads in VPC mode
- Similar to vpc mode, but even more restrictive. In addition to resource being in the VPC, any host or resource
trying to access your cluster must also have the sufficient IAM (Identity and Access management) permissions. You can
grant these permission to your workloads in 2 ways:
- configuring
allowAccessTo
on your workload (functions, container workloads, batch jobs) - configuring
iamRoleStatements
on your workload (functions, container workloads, batch jobs)
- configuring
- Additionally, IP addresses configured in
whitelistedIps
can also access the cluster (even from the internet). To disable this behavior, and enforce cluster isolation, you can set theforceDisablePublicIp
property.
resources:myRedisCluster:type: redis-clusterproperties:nodeInstanceSize: cache.t3.microdefaultUserPassword: $Secret('redis.password')accessibility:accessibilityMode: scoping-workloads-in-vpc
Pricing
You are charged for:
- Computing nodes, shards and replica nodes:
- Price depends on the instance size, number of shards and number of replica nodes.
- Can be calculated as number of shards * (number of replica nodes + 1 (there is always primary)) * price for node instance
- EXAMPLE 1: Cheapest non-sharded cluster using
cache.t3.micro
instance with 0 replicaNodes: 1 * (0+1) * $0.017 = $0.017/hour (~$12.50/month) - EXAMPLE 2: Sharded cluster with 2 shards using
cache.t3.micro
and 1 replicaNode: 2 * (1+1) * $0.017 = $0.068 per hour (~$49/month)
- Data transfer:
- In most cases there should be no additional cost.
- Backups:
- If retained for at most 1 day: free
- $0.085/GB/month after that
Free Tier (eligible for first 12 months)
- Includes 750 hours of
t2.micro
instances (t3.micro
for the regions in whicht2.micro
is not available) each month.
To learn more about pricing, refer to AWS pricing page
API Reference
No description
Type: string "redis-cluster"
Instance size for every node in the cluster (both primary node and replica nodes, if configured)
Type: string ENUM
Possible values: cache.m4.10xlargecache.m4.2xlargecache.m4.4xlargecache.m4.largecache.m4.xlargecache.m5.12xlargecache.m5.24xlargecache.m5.2xlargecache.m5.4xlargecache.m5.largecache.m5.xlargecache.m6g.12xlargecache.m6g.24xlargecache.m6g.2xlargecache.m6g.4xlargecache.m6g.largecache.m6g.xlargecache.r4.16xlargecache.r4.2xlargecache.r4.4xlargecache.r4.8xlargecache.r4.largecache.r4.xlargecache.r5.12xlargecache.r5.24xlargecache.r5.2xlargecache.r5.4xlargecache.r5.largecache.r5.xlargecache.r6g.12xlargecache.r6g.24xlargecache.r6g.2xlargecache.r6g.4xlargecache.r6g.largecache.r6g.xlargecache.t2.mediumcache.t2.microcache.t2.smallcache.t3.mediumcache.t3.microcache.t3.small
- Different instance sizes have different memory and network capabilities.
- Instance size can be changed without interupting the cluster or losing the data.
- To learn more, and see a detailed list of available instances, refer to AWS pricing page
Password for the default cluster user.
Type: string
- Redis clusters are password protected. To communicate with the cluster, you must configure a password for the default user.
- Communication with the cluster is encrypted in-transit.
- You should not input the password directly. The recommended way is using a
.env
file together with File directive or even better, using secrets. - Password constraints:
- Must be only printable ASCII characters.
- Must be at least 16 characters and no more than 128 characters in length.
- Cannot contain any of the following characters: '/', '"', or '@'.
Configures number of shards for the cluster
Type: number
- If you specify more than 1 shard, the cluster will run in the sharded mode
- Sharding enables you to scale redis cluster horizontally, by splitting the data into shards.
- Each shard is then served by a shard cluster, which is responsible for given shard.
Each shard cluster has a primary instance and
numReplicaNodes
amount of replicas. - You can update (increase or decrease) number of shards without interupting the cluster.
- Routing to correct shard and shard rebalancing is automatically handeled by AWS.
- To learn more about sharding, refer to AWS Docs.
Sharding comes with multiple limitations:
- Sharding can only be enabled (set to more than 1) during redis cluster initial creation. After that the cluster cannot be updated from non-sharded to sharded or vice-versa.
- Changing
numReplicaNodes
when sharding is enabled is not possible. In this case it is necessary to delete and recreate the cluster. numReplicaNodes
must be set to at least1
when sharding is enabled.
Number of replica nodes in the cluster
Type: number
- Adding replica nodes (read replicas) to a cluster has 2 benefits:
- increases the read throughput
- increases availability. Replica node can become the primary node in case of primary node failure.
- Load balancing between replicas is automatically handeled by AWS.
- When there are multiple shards, this number specifies the number of replicas for each shard cluster.
Changing numReplicaNodes
when sharding is enabled is not possible and will result in error.
Configures logging behavior and logging format
Type: RedisLogging
- Slow log is sent to a pre-created Cloudwatch log group.
- By default, logs are retained for 180 days.
- You can browse your logs in 2 ways:
- go to the log group page in the AWS CloudWatch console. You can use
stacktape stack-info
command to get a direct link. - use stacktape logs command to print logs to the console
- go to the log group page in the AWS CloudWatch console. You can use
Configures the retention period for automated backups
Type: number
- When set to 0, automatic backups are disabled.
- You can also take manual backup snapshots (in the AWS console or using the API). The retention period is not applied to manual backup snapshots.
Port on which the redis-cluster should listen for connections
Type: number
Configures which resources and hosts can access your cluster
Type: RedisAccessibility
Overrides one or more properties of the specified child resource.
Type: Object
- Child resouces are specified using their descriptive name (e.g.
DbInstance
orEvents.0.HttpApiRoute
). - To see all configurable child resources for given Stacktape resource, use
stacktape stack-info --detailed
command. - To see the list of properties that can be overriden, refer to AWS Cloudformation docs.