does rudder scale linearly if i simply deploy more nodes? i.e for 50k inserts/sec deploy ~ 35 nodes? @glamorous-shoe-15731

It does but there are a couple of things to consider 1. For some downstream destinations event ordering matters - for example, you cannot send a track call before an identity call to Hubspot. If you just setup multiple nodes with a load-balancer in front, the event order may be messed up. What you need to ensure is that events from a given user_id/anonymous_id are all sent to the same Rudder node. Furthermore, if you add/remove a node, the events are rebalanced to ensure this is preserved. This event-ordering/rebalancing feature is part of the enterprise rudder offering. However, if you don't care about event ordering, e.g. if you are just dumping into S3), then yes multiple independent Rudder nodes would work great 2. For warehouse destinations we need some coordination between nodes - Rudder updates the schema if a new field is added. If multiple Rudder tries to do that, there would be a conflict. Some warehouses (e.g. Redshift) also have issues if multiple copy commands are run against the same namespace simultaneuosuly. That's again handled in our enterprise multi-node offering. Not sure if that's a problem with clickhouse. If not, it should work

Thanks for the detailed answer 👍

You are welcome 🙂

how can i enable consent management with rudderstack? i.e something like this https://github.com/segmentio/consent-manager

Hi, can I get some help for the rudder server installation ? Link here

the field context_user_agent seems to be missing for events fired from node-sdk (under tracks table). is this intentional?

For hosted control plane, how do i secure the server? database credentials are listed inside the config.json

Using the rudderstack NodeJS sdk latest version I'm getting duplicated columns. For example for this call :

client.identify({
    anonymousId: 'wowowowowwo',
    traits: {
      apple: 'orange'
    }
  })

This results in 2 columns in

identifies

table, column

apple

and

context_traits_apple

Why is this the case ?

Hi, I want to do a developer machine setup, and the last command

go run -mod=vendor main.go

gives

build context: cannot load context: open /home/john/rudder-server/vendor/context: no such file or directory

->Has anyone succeed in doing that setup ? I am using

ubuntu 20.04

,

node 10.19.0

,

go 1.13.8

,

posqres 13.1

Hi guys, I have added FullStory as a destination in RudderStack. But nothing happens. I have changed

window['_fs_debug'] = true

, but I still cannot debug on FullStory. Do I have to add

FS.identify

function to my scripts? Please help me. Currently, I use Rudderstack self-hosted. Many thanks.

On android sdk once an identify call is made, all subsequent track calls will include traits defined in the first identify call. How to enable this on javascript SDK? The only documentation I can find is https://docs.rudderstack.com/rudderstack-sdk-integration-guides/rudderstack-javascript-sdk#5-2-trait But doing it that way I still have to manually pass the traits. Is there a workaround?

Events from sdk to rudderstack suddenly stopped coming in. How do i debug this? Any pointers? Just happened after deployment using the latest version on docker

Hi, I am now trying to use the docker setup with the config generator. When I do a simple script, nothing is happening. Here is my rudder-docker.yml:

Hi Team, I am using golang sdk 1. How can I check if an event batch was successfully sent or not and take required actions accordingly? 2. If an event within a batch is failed, what happens to the rest of events?

@freezing-agent-68538 thanks for joining the webinar and asking your question. i pinged @narrow-rain-89253 with more context - i think it would be helpful if you asked your questions in a thread here so that other open source users can learn as well

Hi all! I'm having a login issue on the site right now. Seeing this in the console:

Access to XMLHttpRequest at '<https://api.rudderstack.com/login>' from origin '<https://app.rudderstack.com>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

@curved-computer-12942 sorry about that. looping in @narrow-rain-89253 and @agreeable-army-26912 on this

how does rudderstack determine when to send events to warehouse? I have set both

warehouseSyncFreqIgnore

to true, and

uploadFreqInS

to 15 , but it's sending events every 30 minutes to my warehouse

Hi all, I've installed Rudderstack following the official doc (https://docs.rudderstack.com/installing-and-setting-up-rudderstack/native-installation). Almost everything is working well however the fields

traits

and

context

do not appear on the events. We tried to push the same events using the Rudderstack API (https://hosted.rudderlabs.com) and there the fields appears, any idea on what could be the problem? thanks Event push through our self hosted api:

{
  "anonymousId": "616ba46c-a53b-4cea-955e-7fbfa2411a0b",
  "messageId": "node-e80f5306d730b3597a18b43f3a528792-1cdd36eb-037f-47ba-a2bb-a3bb7a25119c",
  "originalTimestamp": "2021-01-08T14:58:55Z",
  "sentAt": "2021-01-08T14:58:56.366Z",
  "type": "identify",
  "userId": "node-poc-admin-1"
}

Same event going throuth the rudderstack api

{
    "_metadata": {
        "nodeVersion": "12.6.0"
    },
    "context": {
        "app": "node-poc",
        "library": {
            "name": "analytics-node",
            "version": "0.0.3"
        },
        "traits": {
            "email": "xxx@xxx.xx",
            "name": "John Doe",
            "role": "admin"
        }
    },
    "messageId": "node-c33a1402368e71c7a29093e2c1e4c7a5-e0c13592-2f1a-4f0a-88b0-bd69fa75e5ac",
    "originalTimestamp": "2021-01-08T15:00:47.305Z",
    "receivedAt": "2021-01-08T15:00:46.916Z",
    "rudderId": "6b37dda5-bbb1-4d0e-b0f8-d898bfae8af1",
    "sentAt": "2021-01-08T15:00:47.306Z",
    "traits": {
        "email": "xxx@xxx.xx",
        "name": "xxx",
        "role": "admin"
    },
    "type": "identify",
    "userId": "node-poc-admin-1"
}

Guys, Rudder backend crashes randomly, how can I debug?

Here is the crash message:

{
   "StartTime": 1609664177,
   "CrashTime": 1609972336,
   "ReadableStartTime": "2021-01-03 08:56:17 +0000 UTC",
   "ReadableCrashTime": "2021-01-06 22:32:16 +0000 UTC",
   "Message": "pq: could not resize shared memory segment \"/PostgreSQL.1947790853\" to 4194304 bytes: No space left on device",
   "StackTrace": "goroutine 1157 [running]:\<http://ngithub.com/rudderlabs/rudder-server/utils/misc.RecordAppError(0x1b92de0|ngithub.com/rudderlabs/rudder-server/utils/misc.RecordAppError(0x1b92de0>, 0xc00014ca50)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/utils/misc/misc.go:119 +0xa0\<http://ngithub.com/rudderlabs/rudder-server/rruntime.Go.func1.1(0x1bbd460|ngithub.com/rudderlabs/rudder-server/rruntime.Go.func1.1(0x1bbd460>, 0xc000ba5e00)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/rruntime/goroutine-factory.go:41 +0x2c5\npanic(0x15b1a40, 0xc000df2d80)\n\t/root/.goenv/versions/1.13.8/src/runtime/panic.go:679 +0x1b2\<http://ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).assertError|ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).assertError>(0xc0006a0200, 0x1b944a0, 0xc000df2d80)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/jobsdb/jobsdb.go:212 +0xe6\<http://ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).getUnprocessedJobsDS(0xc0006a0200|ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).getUnprocessedJobsDS(0xc0006a0200>, 0xc000166be0, 0xb, 0xc002437d40, 0x11, 0xc000166be8, 0x3, 0xc0005c2ab0, 0x1, 0x1, ...)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/jobsdb/jobsdb.go:1762 +0x552\<http://ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).GetUnprocessed(0xc0006a0200|ngithub.com/rudderlabs/rudder-server/jobsdb.(*HandleT).GetUnprocessed(0xc0006a0200>, 0xc0005c2ab0, 0x1, 0x1, 0x2710, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/jobsdb/jobsdb.go:2725 +0x5c0\<http://ngithub.com/rudderlabs/rudder-server/router.(*HandleT).generatorLoop(0xc000dba1e0)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/router/router.go:940|ngithub.com/rudderlabs/rudder-server/router.(*HandleT).generatorLoop(0xc000dba1e0)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/router/router.go:940> +0x7fd\<http://ngithub.com/rudderlabs/rudder-server/router.(*HandleT).Setup.func3()\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/router/router.go:1086|ngithub.com/rudderlabs/rudder-server/router.(*HandleT).Setup.func3()\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/router/router.go:1086> +0x4d\<http://ngithub.com/rudderlabs/rudder-server/rruntime.Go.func1(0xc000764300)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/rruntime/goroutine-factory.go:46|ngithub.com/rudderlabs/rudder-server/rruntime.Go.func1(0xc000764300)\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/rruntime/goroutine-factory.go:46> +0x81\ncreated by <http://github.com/rudderlabs/rudder-server/rruntime.Go\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/rruntime/goroutine-factory.go:32|github.com/rudderlabs/rudder-server/rruntime.Go\n\t/codebuild/output/src878603763/src/github.com/rudderlabs/rudder-server/rruntime/goroutine-factory.go:32> +0x3f\n",
   "Code": 101
  },

How can I fix it?

@wooden-notebook-24208 are you running the Docker version? If yes, the error reported typically occurs because Docker restricts shared memory size to 64 MB. Can you try using the option --shm-size with docker run? Say docker run --shm-size=1g

Yes, I did this instead, let me know if it should fix it.

db:
    image: postgres:11-alpine
    shm_size: 1gb

You can set this in docker-compose as well

Yes. This works as well

then

docker-compose  down

and

docker-compose up -d

is the right thing to do or should I build anew?

Build should not be required

Guys, is Rudder Moengage integration release happening anytime soon? cc @some-hairdresser-58556

@freezing-agent-68538 We support MoEngage. Here is the doc: https://docs.rudderstack.com/destinations/moengage