Hey guys :waving-from-afar-left: When building th...
# troubleshoot
g
Hey guys waving from afar left When building the project (with
./gradlew build
) I keep getting stuck at 99% due to the installation of dependencies (with pip). It seems to have difficulties with
docutils
and
dill
from metadata ingestioninstallDev. Should the have more strict version intervals in
metadata-ingestion/setup.py
or something of that sort? Am I doing something wrong? Screenshot from my 4th attempt
h
it's a long story but try pulling the latest changes from the main branch
🙏 1
🙏 1
g
Giving it try as we speak. I will let you know if it worked soon. Thank you for your input 🙌
Thank you @high-hospital-85984 Now it raises another exception 😕
Copy code
* What went wrong:
Execution failed for task ':datahub-web-react:yarnTest'.
I'm using
Java 8
and yarn
1.22.0
. Do you have any idea of what it can be?
h
there's probably more info in the output?
g
Copy code
removed
h
So clearly some test failed, but that print out does not really tell which one 🤔
g
This is the print of the stacktrace. I think I found the one who failed:
Copy code
FAIL src/app/entity/shared/containers/profile/__tests__/EntityProfile.test.tsx (52.08 s)
  ● EntityProfile › renders dataset page

    thrown: "Exceeded timeout of 5000 ms for a test.
    Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

      23 |
      24 | describe('EntityProfile', () => {
    > 25 |     it('renders dataset page', async () => {
         |     ^
      26 |         const { getByText } = render(
      27 |             <MockedProvider mocks={mocks} addTypename={false}>
      28 |                 <TestPageContainer initialEntries={['/dataset/urn:li:dataset:3']}>

      at src/app/entity/shared/containers/profile/__tests__/EntityProfile.test.tsx:25:5
      at Object.<anonymous> (src/app/entity/shared/containers/profile/__tests__/EntityProfile.test.tsx:24:1)
Different things seem to be failing depending on the execution...I'm confused O.o
Task :metadata-io:test FAILED
Copy code
com.linkedin.metadata.graph.ElasticSearchGraphServiceTest.setup FAILED
    org.testcontainers.containers.ContainerLaunchException at ElasticSearchGraphServiceTest.java:43
        Caused by: org.rnorth.ducttape.RetryCountExceededException at ElasticSearchGraphServiceTest.java:43
            Caused by: org.testcontainers.containers.ContainerLaunchException at ElasticSearchGraphServiceTest.java:43
                Caused by: org.testcontainers.containers.ContainerLaunchException at ElasticSearchGraphServiceTest.java:43
Another one:
> Task :metadata-ingestion:lint FAILED
Copy code
return client

     def get_workunits(self) -> Iterable[MetadataWorkUnit]:
         client = self._get_looker_client()
         dashboards = client.all_dashboards(fields="id")
-        deleted_dashboards = client.search_dashboards(fields="id", deleted="true") if self.source_config.include_deleted else []
+        deleted_dashboards = (
+            client.search_dashboards(fields="id", deleted="true")
+            if self.source_config.include_deleted
+            else []
+        )
         dashboard_ids = [
             dashboard_base.id
             for dashboard_base in dashboards + deleted_dashboards
             if dashboard_base.id is not None
         ]
@@ -410,11 +414,17 @@
             self.reporter.report_dashboards_scanned()
             if not self.source_config.dashboard_pattern.allowed(dashboard_id):
                 self.reporter.report_dashboards_dropped(dashboard_id)
                 continue
             try:
-                fields = ["id", "title", "dashboard_elements", "dashboard_filters", "deleted"]
+                fields = [
+                    "id",
+                    "title",
+                    "dashboard_elements",
+                    "dashboard_filters",
+                    "deleted",
+                ]
                 dashboard_object = client.dashboard(
                     dashboard_id=dashboard_id, fields=",".join(fields)
                 )
             except SDKError:
                 # A looker dashboard could be deleted in between the list and the get
would reformat src/datahub/ingestion/source/looker.py
Oh no! 💥 💔 💥 1 file would be reformatted, 215 files would be left unchanged.
@green-football-43791 sorry for the trouble, but can you please advise?
g
Hey there- so, just to understand, you are seeing different errors each time you build?
g
yes, that's right 😕 Or a combination of the above on the same execution.
g
got it. what happens if you cd into datahub-web-react and run
yarn test
?
g
No tests found related to files changed since last commit. But I run the test pressing
a
and then starts throwing:
Copy code
console.error
      Could not fetch logged in user from cache. + No more mocked responses for the query: query getMe {
        me {
          corpUser {
            urn
            username
            info {
              active
              displayName
              title
              firstName
              lastName
              fullName
              email
              __typename
            }
            editableInfo {
              pictureLink
              teams
              skills
              __typename
            }
            __typename
          }
          platformPrivileges {
            viewAnalytics
            managePolicies
            __typename
          }
          __typename
        }
      }
      , variables: {}
g
gotcha, those are test warnings
g
and also:
Copy code
console.error
      Warning: validateDOMNesting(...): <a> cannot appear as a descendant of <a>
Test Suites: 31 passed, 31 total
Tests:       81 passed, 81 total
Snapshots:   0 total
Time:        46.54 s, estimated 96 s
Ran all test suites.
g
great, sounds like the above test was just failing due to resource issues
that is also likely the issue with
:metadata-io:test
I have noticed the ElasticSearchGraphServiceTest fails if run without sufficient resources
the
:metadata-ingestion:lint
looks real though... let me look into that
g
FYI I'm using
macOS 11.5.2
Processor
2,2 GHz Quad-Core Intel Core i7
Memory
16 GB 1600 MHz DDR3
Docker setup CPUs 4 Memory 10GB Swap: 3GB Disk Image Size 80Gb
Thank you for your time @green-football-43791 🙏
g
Got it, thanks for the docker resource information
🙏 1
I see the lint issue too- looks like this was a regression in master!
not sure how this snuck through CI 😕
let me fix this
g
These things happen 🙂 I would appreciate that, Gabe. Thanks 🙂
g
Here's a PR to fix the lint issue:
g
Thank you for the quick fix. Will be waiting for the merge to give it a try. Will let you know if it works...let's hope 🙌
Good and bad news @green-football-43791. Good: No more lint errors 🙂
Test Suites: 1 failed, 30 passed, 31 total
Tests:       2 failed, 79 passed, 81 total
Snapshots:   0 total
Time:        161.628 s
Bad: The timeouts issues remain 🙃
Copy code
Summary of all failing tests
FAIL src/app/search/__tests__/SearchPage.test.tsx (72.202 s)
  ● SearchPage › renders loading

    thrown: "Exceeded timeout of 5000 ms for a test.
    Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

      11 |
      12 | describe('SearchPage', () => {
    > 13 |     it('renders loading', async () => {
         |     ^
      14 |         const promise = Promise.resolve();
      15 |         const { getByText } = render(
      16 |             <MockedProvider mocks={mocks} addTypename={false}>

      at src/app/search/__tests__/SearchPage.test.tsx:13:5
      at Object.<anonymous> (src/app/search/__tests__/SearchPage.test.tsx:12:1)

  ● SearchPage › renders multiple checked filters at once

    thrown: "Exceeded timeout of 5000 ms for a test.
    Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

      53 |     });
      54 |
    > 55 |     it('renders multiple checked filters at once', async () => {
         |     ^
      56 |         const promise = Promise.resolve();
      57 |         const { getByTestId, queryByTestId } = render(
      58 |             <MockedProvider mocks={mocks} addTypename={false}>

      at src/app/search/__tests__/SearchPage.test.tsx:55:5
      at Object.<anonymous> (src/app/search/__tests__/SearchPage.test.tsx:12:1)
Are you aware of something that can be done or should I only try to assess the possibility of developing in another machine? Nevertheless, maybe the requirements information should be updated. What do you think? Thank you again for your time and the quick fix. You rock!! 🚀
g
hmm, for now as a quick workaround
you can run
./gradlew build -x yarnTest -x test
that will skip the tests during build
g
Thank you so much! Gave it another try, and was able to do it. It should be memory allocation related issues, as you said.
Again, I can't thank you enough 🙂
g
Nice!!
Glad it worked 🎉
🙌 1
Separately, I'll look into removing these memory intensive tests from the main build
g
That would be nice for someone with not so great PCs 😇
2