Hi Community. Is there a way to create datasets using the Java Emitter? We are looking to create datasets using the Java Emitter programmatically. Any help appreciated. Thanks in advance!

Hey, Yes it does possible, see some example code here: https://github.com/datahub-project/datahub/blob/c7f477813c93fa73811f37614cf655808eed353a/metadata-integration/java/as-a-library.md#usage

Hey Peter! Thanks for responding. I already took a look at this. My use case is to build the dataset using the emitter. I've tried something like this. Does this seem to be the right approach or is there a better approach?

SchemaField field  = new SchemaField();
        field.setDescription("Field 1 description");
        field.setLabel("table_via_emitter");
        field.setFieldPath("field1");
        field.setType(new SchemaFieldDataType().setType(SchemaFieldDataType.Type.create(new NullType())));
        field.setNativeDataType("String");


        SchemaFieldArray schemaFieldArray = new SchemaFieldArray();
        schemaFieldArray.add(field);

        DatasetUrn datasetUrn = new DatasetUrn(new DataPlatformUrn("db2"),"table_via_emitter", FabricType.DEV);
        SchemaMetadata schemaMetadata = new SchemaMetadata();
        schemaMetadata.setDataset(datasetUrn);
        schemaMetadata.setFields(schemaFieldArray);
        schemaMetadata.setSchemaName("test_schema");
        schemaMetadata.setPlatformSchema(SchemaMetadata.PlatformSchema.create(new KeyValueSchema().setKeySchema("key").setValueSchema("value")));
        schemaMetadata.setVersion(1);
        schemaMetadata.setHash("hash");
        schemaMetadata.setPlatform(new DataPlatformUrn("db2"));


        MetadataChangeProposalWrapper mcpw = MetadataChangeProposalWrapper.builder()
                .entityType("dataset")
                .entityUrn(datasetUrn)
                .upsert()
                .aspect(schemaMetadata)
                .aspectName("schemaMetadata")
                .build();

Can I ask about your usecase? Is your source not supported?

Yeah our data source is not supported and we are exploring ways other than sqlalchemy for comparing the possibilities..

I see, if it's a source that is commonly used, and you want to contribute to back, we are happy to take it 🙂

okay, thanks! 🙂