When using S3 as the deepstore with the SegmentCreationAndMetadataPush, should the

controller.data.dir

(from the controller conf) be the same as the

outputDirURI

(from the ingestion jobspec) ?

i’ve tried this and you can have:

outputDirUri for ingestion job 1:
<s3://bucket_name/controller_data_dir/output_dir_uri_1>

outputDirUri for ingestion job 2:
<s3://bucket_name/controller_data_dir/output_dir_uri_1>

controller.data.dir
<s3://bucket_name/controller_data_dir/>

i.e.

controller.data.dir

works as the root dir.

thanks! I currently have them as different dirs and it seems to work too. From my understanding, the controller.data.dir is only used when doing the tar push instead of the metadata push?