Hi, I just raised a PR for adding a new ingestion ...
# contribute-code
b
Hi, I just raised a PR for adding a new ingestion source. One of the checks (Vercel) is failing and I don’t have access to see why - is that something that will be investigated when the PR is looked at? Or should I be asking in here for someone to look at now? Thanks https://github.com/datahub-project/datahub/pull/7707
1
a
Hi, the vercel check has to do with the docs website- did you add/build documentation in the pr? It may also be a failure due to an untracked markdown file somewhere
b
Are you able to be more specific about the error, or post the check output? There is documentation in the pr which builds successfully locally with
./gradlew :docs-website:build
Oh hold on, it builds successfully but I’m not seeing the docs when I run
npm run serve
Which is weird, because running
./gradlew :metadata-ingestion:docGen
completes successfully and generates my docs
docGen completes successfully but with these errors, not sure how significant they are:
Copy code
logger - ERROR - Deequ is still not supported in spark version: 3.2.0
py.warnings - WARNING - /Users/gmcgoldrick/Repositories/rapid7-datahub/datahub/metadata-ingestion/venv/lib/python3.10/site-packages/tableschema/profile.py:91: DeprecationWarning: Subclassing validator classes is not intended to be part of their public API. A future version will make doing so an error, as the behavior of subclasses isn't guaranteed to stay the same between releases of jsonschema. Instead, prefer composition of validators, wrapping them in an object owned entirely by the downstream library.
  class _TableSchemaValidator(validator_for(_PROFILES['table-schema'])):

__main__ - ERROR - Failed to find source classes for platform elastic-search. Did you remember to annotate your source class with @platform_name(elastic-search)?
Ingestion Documentation Generation Complete
############################################
{
  "source_platforms": {
    "discovered": 52,
    "generated": 51,
    "warnings": [
      "Failed to find source classes for platform elastic-search. Did you remember to annotate your source class with @platform_name(elastic-search)?"
    ]
  },
  "plugins": {
    "discovered": 57,
    "generated": 57,
    "failed": 0
  }
}
############################################

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/6.9.2/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD SUCCESSFUL in 1m 53s
28 actionable tasks: 10 executed, 18 up-to-date
I’m not sure which of the source_platforms it doesn’t generate. It does definitely generate mine at
./docs/generated/ingestion/sources/amplitude.md
Ok, I can now see my docs at localhost:3000 after running
npm run serve
But I didn’t change anything in the code
a
Odd indeed! And the platform errors aren’t persisiting?
b
gradlew metadata ingestiondocGen
This is the output from
./gradlew :metadata-ingestion:docGen
☝️
gradlew docs websitebuild
And this is from the subsequent
./gradlew :docs-website:build
command ☝️
This is built from my branch that is up to date with master. The only errors I can see in there are from pip and a SPARK_VERSION env var (which are local and don’t effect the build)
It’s hard to debug the issue without seeing the output from Vercel
I should add that the above outputs both show build successful and I can see the docs locally
a
@gray-shoe-75895, any ideas here- I’m unsure why vercel would be failing on a local build
b
Hi, would it be possible to get an update on this please? It really would help if I could get the log from Vercel on why it’s failing
g
Looking at the logs, it doesn’t like this line
Copy code
projects: list[AmplitudeProjectConfig]
probably need to use typing.List, because we still support back to python 3.7
The main CI should also report the errors/warnings on that as well. However, because you’re a first time contributor, it requires approval to run the normal CI
b
Thanks so much for this 😄
Just a heads-up that Vercel is happy now, appreciate the help. Would it be possible to get the pr looked at? Thanks!