Good Morning Everyone, I hope there is a solution...
# help
m
Good Morning Everyone, I hope there is a solution to the problem I have been struggling with. I have a Global Dynamodb Table and I want to run a stream in each of the regions that the table is in (It sends data to an Elastic instance in each region) However nothing I have tried works. I am wondering if I am taking the wrong approach:
Copy code
if(scope.region === "us-east-2") {
    const replicationRegions = scope.local ? [] : ["ap-southeast-1"]
    this.globalchartflowdata = new sst.Table(this, "GlobalChartflowData", {
        fields: {
            pk: "string",
            sk: "string",
            gsi1pk: "string",
            gsi1sk: "string",
        },
        primaryIndex: {partitionKey: "pk", sortKey: "sk"},
        globalIndexes: {
            "GSI1": {partitionKey: "gsi1pk", sortKey: "gsi1sk"},
        },
        stream: true,
        consumers: {
            dynamodbstream: {
                function: {
                    srcPath: "backend/",
                    handler: "services/dynamodbstream/stream.handler",
                    environment: {
                        STAGE: props.rootStage,
                    },
                    permissions: [new iam.PolicyStatement({
                        actions: ["ssm:GetParameter", "ssm:GetParameters", "ssm:GetParametersByPath"],
                        resources: ["*"],
                    })]
                }
            }
        },
        cdk: {
            table:
            {
                replicationRegions: replicationRegions,
            }
        }

    })
}
else{
    const tableArn = `arn:aws:dynamodb:${scope.region}:309833148800:table/${props.rootStage}-chartflow-GlobalChartflowData`
    this.globalchartflowdata = new sst.Table(this, "GlobalChartflowData", {
      cdk: { table: dynamodb.Table.fromTableArn(this, "ImportedTable", tableArn),
    }});
    this.globalchartflowdata.stream = true;
    this.globalchartflowdata.addConsumers(this, {dynamodbstream: {
        function: {
            srcPath: "backend/",
            handler: "services/dynamodbstream/stream.handler",
            environment: {
                STAGE: props.rootStage,
            },
            permissions: [new iam.PolicyStatement({
                actions: ["ssm:GetParameter", "ssm:GetParameters", "ssm:GetParametersByPath"],
                resources: ["*"],
            })]
        }
    }})
}
In the else clause I have tried several variations, including not setting stream = true and trying to set stream = true in the new sst.Table, neither of which work. I either get:
Copy code
Error: Please enable the "stream" option to add consumers to the "GlobalChartflowData" Table. To import a table with stream enabled, use the "Table.fromTableAttributes()" method, and set the "tableStreamArn" in the attributes.
    at Table.addConsumer (file:///tmp/seed/source/node_modules/@serverless-stack/resources/dist/Table.js:283:19)
    at file:///tmp/seed/source/node_modules/@serverless-stack/resources/dist/Table.js:164:18
    at Array.forEach (<anonymous>)
    at Table.addConsumers (file:///tmp/seed/source/node_modules/@serverless-stack/resources/dist/Table.js:163:32)
    at new DynamodbStack (file:///tmp/seed/source/.build/lib/index.js:88:32)
    at Module.main (file:///tmp/seed/source/.build/lib/index.js:364:24)
    at file:///tmp/seed/source/.build/run.mjs:92:16
or I get:
Copy code
Error: Cannot configure the "stream" when "cdk.table" is a construct in the "Table" Table
    at Table.createTable (file:///tmp/seed/source/node_modules/@serverless-stack/resources/dist/Table.js:231:23)
    at new Table (file:///tmp/seed/source/node_modules/@serverless-stack/resources/dist/Table.js:29:14)
    at new DynamodbStack (file:///tmp/seed/source/.build/lib/index.js:82:34)
    at Module.main (file:///tmp/seed/source/.build/lib/index.js:364:24)
    at file:///tmp/seed/source/.build/run.mjs:92:16
anyone have any ideas? I did notice this Stack Overflow Question that seems somewhat similar, but it is talking about a resource completely outside of CDK: https://stackoverflow.com/questions/65557316/aws-cdk-working-with-existing-dynamodb-and-streams but it didn't make a lot of sense to me. I tried just putting the table definition in the else clause without the replication, but that just created a second table in the region with a different name: dev-chartflow-GlobalChartflowData (Replicant Table without the Stream) dev-singapore-chartflow-GlobalChartflowData (New Local Table with the Stream)
f
Hey @Michael Robellard, in the non root stage (ie. in the else block), instead of importing the table like this:
Copy code
dynamodb.Table.fromTableArn(this, "ImportedTable", tableArn),
change it to
Copy code
dynamodb.Table.fromTableAttributes(this, "ImportedTable", {
  tableArn,
  tableStreamArn,
}),
As the suggested in the error message you shared above:
Copy code
Error: Please enable the "stream" option to add consumers to the "GlobalChartflowData" Table. To import a table with stream enabled, use the "Table.fromTableAttributes()" method, and set the "tableStreamArn" in the attributes.
When importing tables with stream already enabled, SST needs to know the stream’s ARN in order to add consumers.
You can find the stream ARN in the DynamoDB console
Let me know if that makes sense
m
You know I must have read that error message a dozen times and said to myself that is exactly what I am doing, thinking it said fromTableArn and set the tableArn.
That seemed to work.
Is there a better way to handle the arns than just manually looking them up and adding them to the script?
Copy code
const arns = {
    "us-east-2": {
        "dev": {
            global: "arn:aws:dynamodb:us-east-2:309833148800:table/dev-chartflow-GlobalChartflowData/stream/2022-02-03T22:54:27.554",
            lookup: "arn:aws:dynamodb:us-east-2:309833148800:table/dev-chartflow-LookupChartflowData/stream/2022-02-03T22:54:27.624"
        }
    },
    "ap-southeast-1": {
        "dev": {
            global: "arn:aws:dynamodb:ap-southeast-1:309833148800:table/dev-chartflow-GlobalChartflowData/stream/2022-02-07T16:20:33.637",
            lookup: "arn:aws:dynamodb:ap-southeast-1:309833148800:table/dev-chartflow-LookupChartflowData/stream/2022-05-27T18:10:42.734"
        }
    }
}
Seems a little wrong to have to manually change this everytime I add a new region. Granted I shouldn't need to do that very often, but still seems ugly.