hey <@U088WCP3B4P> I'm running edr report as part ...
# ask-elementary-ai
l
hey @kapa.ai I'm running edr report as part of a databricks workflow, after setting target-path and profiles-dir and passing environment variables for where to install dbt packages to the cluster the edr report command runs fine, however it seems to be hanging indefinitely and neither succeeding nor failing. How can i fix this?
k
A message from kapa.ai
When running edr report as part of a Databricks workflow, if the command hangs indefinitely (neither succeeding nor failing), there are a few common causes and troubleshooting steps based on similar issues reported by the community: 1. Permissions and Working Directory: Elementary tries to create a target directory (like edr_target) for its internal files. If the current working directory is not writable (e.g., inside a cloned repo or a restricted path in Databricks), this can cause the process to hang or fail. Solution: Before running any edr command, change the working directory to a location that is writable (such as /tmp). This ensures Elementary can create its required folders and files. Example workaround:
Copy code
python    import os    os.chdir("/tmp")  # Switch to a writable directory before running edr    # Now run your edr command
This approach was confirmed to resolve similar issues in Databricks Workflows, as the Config class in Elementary tries to create directories before command-line arguments like --target-path are processed. If you don't switch to a writable directory, the process may hang or error out early in execution [Databricks Workflows directory issue](https://slack.com/archives/C02CTC89LAX/p1740037553539379). 2. Cluster Access Mode and Permissions: If you are running on a Databricks cluster in "shared" access mode, you may encounter permission errors when Elementary tries to write to certain locations (like the Python site-packages directory). This can cause the process to hang or fail. Solution: Use a "single user" access mode cluster, or ensure that the paths Elementary needs to write to are accessible. See: [Databricks shared cluster permission error](https://github.com/elementary-data/elementary/issues/1584). 3. Resource Constraints or Large Data Volumes: If the report generation is processing a large amount of data (many test results, long days-back window), it may appear to hang due to long-running queries or memory issues. Solution: Try reducing the --days-back or --executions-limit parameters to limit the data processed. This can help determine if the issue is due to data volume [Timeouts and large data](https://slack.com/archives/C02CTC89LAX/p1691169511715969). 4. Check for Silent Errors: Sometimes, errors are not surfaced in the main output but are present in the edr.log file. Solution: Check the edr.log file in your target directory for any hidden errors or stack traces that might indicate the root cause [Checking edr.log](https://slack.com/archives/C02CTC89LAX/p1697659818900659). Summary of steps to try: - Change the working directory to a writable location (e.g., /tmp) before running edr. - Ensure your cluster access mode and permissions allow writing to the necessary paths. - Reduce the data processed by limiting --days-back or --executions-limit. - Check edr.log for additional error details. If after these steps the issue persists, please provide the edr.log output for further troubleshooting.