welcome @able-keyboard-68141!!

Ok folks! Feb 19: DataHub Townhall videos are up! 1. Full 1hr video : 2. React app demo + Tags preview: 3. DataHub at Geotab: 4. New Python ingestion framework: Watch / Like / Subscribe! 🙂

General questions: 1. Once I run an ingest, will it keep things updated or I'll need to schedule it by my own? 2. For the "natural language" part of the metadata, does it rely on SQL Comments (when ingesting from dbs)? 3. If yes (2), there's something, part of the project, to make it easier for the end user to fill it?

for 1) You will need to schedule it using a scheduler like Airflow (@gray-shoe-75895 has a recipe for how to use Airflow)

@gray-shoe-75895 can you take a look at 2)?

Sure thing! For (1), there's a couple sample Airflow DAGs here. For (2), we pull SQL comments from db tables themselves - these will be visible in the schema tab of the UI. There's currently no way to edit the column-level comments, since they would be overwritten the next time the ingestion pipeline runes. Instead, I would suggest using the documents tab to add natural language information alongside a dataset

Hello, my datahub-frontend container isn't starting..docker logs:

...
19:58:16,434 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5efa40fe - Registering current configuration as safe fallback point
19:58:30 [application-akka.actor.default-dispatcher-2] INFO  akka.event.slf4j.Slf4jLogger - Slf4jLogger started
19:58:34 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No scheduled executor is provided to HttpClientFactory, using it's own scheduled executor.
19:58:34 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No callback executor is provided to HttpClientFactory, using it's own call back executor.
19:58:34 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No Compression executor is provided to HttpClientFactory, using it's own compression executor.
19:58:34 [main] INFO  c.l.r.t.h.client.HttpClientFactory - The service 'null' has been assigned to the ChannelPoolManager with key 'noSpecifiedNamePrefix 1138266797 ' 
19:58:41 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No scheduled executor is provided to HttpClientFactory, using it's own scheduled executor.
19:58:41 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No callback executor is provided to HttpClientFactory, using it's own call back executor.
19:58:41 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No Compression executor is provided to HttpClientFactory, using it's own compression executor.
19:58:41 [main] INFO  c.l.r.t.h.client.HttpClientFactory - The service 'null' has been assigned to the ChannelPoolManager with key 'noSpecifiedNamePrefix 1138266797 ' 
ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.8ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.8ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.8ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.8ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
19:58:59 [main] INFO  play.api.Play - Application started (Prod)
19:59:17 [main] INFO  play.core.server.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9001
This application is already running (Or delete /datahub-frontend/play.pid file).
This application is already running (Or delete /datahub-frontend/play.pid file).
This application is already running (Or delete /datahub-frontend/play.pid file).
This application is already running (Or delete /datahub-frontend/play.pid file).
This application is already running (Or delete /datahub-frontend/play.pid file).

As promised, we will be hosting React office hours next week. Please vote for your preferred time below!

Fresh off the press: https://shirshanka.medium.com/linkedin-datahub-project-updates-february-2021-edition-338d2c6021f0

Just wondering if anyone's seen this:

➜  datahub git:(master) ./gradlew build
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: <https://docs.gradle.org/5.6.4/userguide/gradle_daemon.html>.
Daemon will be stopped at the end of the build stopping after processing
Configuration on demand is an incubating feature.

FAILURE: Build failed with an exception.

* Where:
Build file '/home/garylucas/dev/datahub-contrib/datahub/metadata-models-ext/build.gradle' line: 15

* What went wrong:
A problem occurred evaluating project ':metadata-models-ext'.
> Could not resolve all files for configuration ':metadata-models-ext:dataModel'.
   > Could not resolve com.linkedin.datahub-gma:core-models-data-template:0.2.45.
     Required by:
         project :metadata-models-ext > project :li-utils
      > Could not resolve com.linkedin.datahub-gma:core-models-data-template:0.2.45.
         > Could not get resource '<https://linkedin.jfrog.io/artifactory/open-source/com/linkedin/datahub-gma/core-models-data-template/0.2.45/core-models-data-template-0.2.45.pom>'.
            > Could not GET '<https://linkedin.jfrog.io/artifactory/open-source/com/linkedin/datahub-gma/core-models-data-template/0.2.45/core-models-data-template-0.2.45.pom>'. Received status code 401 from server: Unauthorized

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/5.6.4/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD FAILED in 10s

Just forked / cloned off of master. Wondering if something is broken on my local or if other's are seeing this.

@some-crayon-90964 a workaround for now: Here is something that worked for me. in datahub-gma: 

./gradlew build -x test publishToMavenLocal

Followed by a small change to 

datahub/build.gradle

 to set it to the latest version of datahub-gma from your local repo.

git diff build.gradle                                                                                       master
diff --git a/build.gradle b/build.gradle
index eff6dfd2..cd536717 100644
--- a/build.gradle
+++ b/build.gradle
@@ -1,6 +1,6 @@
 buildscript {
   ext.junitJupiterVersion = '5.6.1'
-  ext.gmaVersion = '0.2.45'
+  ext.gmaVersion = '0.2.47'
   ext.pegasusVersion = '28.3.7'

   apply from: './repositories.gradle'

I've manually uploaded 0.2.45 to bintray; you should be unblocked. I'll coordinate with the LI to make sure this doesn't happen again, or if we aren't sure of that I'll re-enable bintray publishing.

Thanks @microscopic-receptionist-23548! If folks are still having issues building, please let us know.

React office hours are starting! They'll be open today 8-10am PST and 3-5pm PST. Join here to ask questions or just hang out and hack on the app: https://meet.google.com/rbr-vbsy-yuy?authuser=1

A quick announcement: we've added support for docker images tagged with the git-short-sha (in addition to

latest

). So you can work with a specific docker image (built off of master) without getting affected by new changes. See this comment for more details. We're also working on cutting a new release (0.7.0) : Issue tracking this is https://github.com/linkedin/datahub/issues/2206

Welcome @modern-exabyte-51323!!

Me approving PR-s today like: https://giphy.com/gifs/naxep4vNBAOL6/embed

<!here>: Reminder this is happening Mar 19th, 7am US PT! See you all there 🙂

Happening now!

nice TownHall today. very informative. Thank you Wolt for presenting

I'll post the slides, mocks and google form for async feedback in a bit!

Townhall video is now up for people who missed it!

Hey Datahub team, we are getting docker errors on a machine that we do security scan by running

./gradlew build

The machine has following setup: Linux ubuntu 5.8.0-44-generic Docker version 20.10.5 build 55c4c88

Also <!here> bumping Office Hours scheduling to your attention, since lots of people were interested in signing up for this.

Heads up! this is happening on Monday 🙂

If you missed the morning session, office hours is back on now!

<!here> The March edition of the DataHub Project update is out! Get in on the latest news 🙂 https://medium.com/datahub-project/linkedin-datahub-project-updates-697f0faddd10?source=friends_link&amp;sk=9888633c5c7219b875125e87a703ec4d

Welcome @thousands-tailor-5575!

Welcome @able-jelly-81126 @abundant-pencil-95380 @strong-carpet-4774!!

Welcome @fast-leather-13054!!!