Hi, anyone getting this error installing

accumulation-tree

package when trying to install

datahub

? ENV:

OS: macos
arch: x86_64
pip v22.2.2
python version: Python 3.8.9

running

python -m pip install --upgrade acryl-datahub

note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for accumulation-tree
  Running setup.py clean for accumulation-tree
Failed to build accumulation-tree
Installing collected packages: accumulation-tree, tdigest, acryl-datahub
  Running setup.py install for accumulation-tree ... error
  error: subprocess-exited-with-error

  × Running setup.py install for accumulation-tree did not run successfully.
  │ exit code: 1
  ╰─> [24 lines of output]
      /Users/home/.virtualenvs/datahub/lib/python3.8/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      running install
      /Users/home/.virtualenvs/datahub/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.14-x86_64-cpython-38
      creating build/lib.macosx-10.14-x86_64-cpython-38/accumulation_tree
      copying accumulation_tree/treeslice.py -> build/lib.macosx-10.14-x86_64-cpython-38/accumulation_tree
      copying accumulation_tree/__init__.py -> build/lib.macosx-10.14-x86_64-cpython-38/accumulation_tree
      copying accumulation_tree/abctree.py -> build/lib.macosx-10.14-x86_64-cpython-38/accumulation_tree
      running build_ext
      skipping 'accumulation_tree/accumulation_tree.c' Cython extension (up-to-date)
      building 'accumulation_tree.accumulation_tree' extension
      creating build/temp.macosx-10.14-x86_64-cpython-38
      creating build/temp.macosx-10.14-x86_64-cpython-38/accumulation_tree
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -Werror=implicit-function-declaration -arch x86_64 -I/Users/home/.virtualenvs/datahub/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -c accumulation_tree/accumulation_tree.c -o build/temp.macosx-10.14-x86_64-cpython-38/accumulation_tree/accumulation_tree.o
      accumulation_tree/accumulation_tree.c:6:10: fatal error: 'Python.h' file not found
      #include "Python.h"
               ^~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> accumulation-tree

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

@mammoth-bear-12532 seems like the accumulation-tree package is required by tdigest, which was added as a dep in https://github.com/datahub-project/datahub/pull/5693

yup.. it was added by me

I don’t think it’s about host/OS, but instead around availability of python development headers. It looks like the stack trace above is using the python version that comes built-in with xcode

hmm, I just needed a streaming percentile library

I looked briefly and didn’t find any - I think it might makes sense disable that functionality until we find/build something

Came here looking for help with this same issue! So, plus1

@millions-market-27106: we're working on a fix for this

just want to added on this I’m having the same issue with latest master branch when I did a fresh

./gradlew :metadata-ingestion:installDev

@gray-airplane-39227: this has been taken out now, so I'm surprised you are seeing this. Are you sure you are on the latest?

@mammoth-bear-12532 Yes, I tried cloning the repo to a new location and start from fresh, I still seeing this in my local when I run

./gradlew :metadata-ingestion:installDev

, it was working fine last week.

try ./gradlew metadata ingestionclean first

@mammoth-bear-12532 getting the same error. I am working on a fork which I synched today. I also ran

./gradlew :metadata-ingestion:clean

. I am building this on a Mac.

@elegant-state-4 - what’s the full error message that you’re seeing? We shouldn’t have accumulator-tree in our deps anywhere

@gray-shoe-75895 find attached the build logs and the console logs.

Looks like a number of packages are being built from source. Some of these don’t have pre-built wheels on arm64 or x86, which means the source build is unavoidable. That said, my previous comment about python package headers is still correct. Are you using the python built-in to macos, or a different version?

@gray-shoe-75895 I tried using Python 3.8.2 and 3.11.0 and got the same error. A colleague of mine tried building Datahub master branch and got the same error. Am I the only person trying to build Datahub on a Mac? Surely other people must have run into this issue before me