gdal

subprocess

LinkageError occurred while loading main class org.apache.spark.launcher.Main java.lang.UnsupportedClassVersionError:

Py4JJavaError: An error occurred while calling <http://None.org|None.org>.apache.spark.api.java.JavaSparkContext. : java.lang.UnsupportedOperationException: getSubject is not supported

/usr/lib/jvm/java-11-openjdk-amd64/

/usr/lib/jvm/jdk-25.0.1-oracle-x64/bin

with KedroSession.create(project_path=project_path,package_name="package", env="end") as session:
    session.run(node_names=["ds1"
                             ])

ds1:
  type: "${globals:datatypes.csv}"
  filepath: "abfss://<container>@<acount_name>.<http://dfs.core.windows.net/raw_data/ds1.csv.gz|dfs.core.windows.net/raw_data/ds1.csv.gz>"
  fs_args:
    account_name: "accountName"
    sas_token: "sas_token"
  layer: raw_data
  load_args:
    sep: ";"
    escapechar: "\\"
    encoding: "utf-8"
    compression: gzip
    #lineterminator: "\n"
    usecols:

DatasetError: An exception occurred when parsing config for dataset 'summary':
No module named 'tracking'. Please install the missing dependencies for 
tracking.MetricsDataset:
<https://docs.kedro.org/en/stable/kedro_project_setup/dependencies.html#install-d>
ependencies-related-to-the-data-catalog
Hint: If you are trying to use a dataset from `kedro-datasets`, make sure that 
the package is installed in your current environment. You can do so by running 
`pip install kedro-datasets` or `pip install kedro-datasets[<dataset-group>]` to
install `kedro-datasets` along with related dependencies for the specific 
dataset group.

Layer.call

kedro package

spark executor

no module named xxx

uvx kedro new --starter spaceflights-pandas --name spaceflights
cd spaceflights

uv run kedro run --pipeline __default__

[11/21/25 00:21:49] INFO     Using 'conf/logging.yml' as logging configuration. You can change this by setting the KEDRO_LOGGING_CONFIG environment variable accordingly.          __init__.py:270
                    INFO     Kedro project spaceflights                                                                                                                             session.py:330
[11/21/25 00:21:51] INFO     Kedro is sending anonymous usage data with the sole purpose of improving the product. No personal data or IP addresses are stored on our side. To opt   plugin.py:243
                             out, set the `KEDRO_DISABLE_TELEMETRY` or `DO_NOT_TRACK` environment variables, or create a `.telemetry` file in the current working directory with the              
                             contents `consent: false`. To hide this message, explicitly grant or deny consent. Read more at                                                                      
                             <https://docs.kedro.org/en/stable/configuration/telemetry.html>                                                                                                        
                    WARNING  Workflow tracking is disabled during partial pipeline runs (executed using --from-nodes, --to-nodes, --tags, --pipeline, and more).                  run_hooks.py:135
                             `.viz/kedro_pipeline_events.json` will be created only during a full kedro run. See issue <https://github.com/kedro-org/kedro-viz/issues/2443> for                     
                             more details.                                                                                                                                                        
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/coder/spaceflights/.venv/lib/python3.13/site-packages/kedro/io/core.py:187 in from_config  │
│                                                                                                  │
│    184 │   │                                                                                     │
│    185 │   │   """                                                                               │
│    186 │   │   try:                                                                              │
│ ❱  187 │   │   │   class_obj, config = parse_dataset_definition(                                 │
│    188 │   │   │   │   config, load_version, save_version                                        │
│    189 │   │   │   )                                                                             │
│    190 │   │   except Exception as exc:                                                          │
│                                                                                                  │
│ /home/coder/spaceflights/.venv/lib/python3.13/site-packages/kedro/io/core.py:578 in              │
│ parse_dataset_definition                                                                         │
│                                                                                                  │
│    575 │   │   │   │   "related dependencies for the specific dataset group."                    │
│    576 │   │   │   )                                                                             │
│    577 │   │   │   default_error_msg = f"Class '{dataset_type}' not found, is this a typo?"      │
│ ❱  578 │   │   │   raise DatasetError(f"{error_msg if error_msg else default_error_msg}{hint}")  │
│    579 │                                                                                         │
│    580 │   if not class_obj:                                                                     │
│    581 │   │   class_obj = dataset_type                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
DatasetError: Dataset 'MatplotlibWriter' not found in 'matplotlib'. Make sure the dataset name is correct.
Hint: If you are trying to use a dataset from `kedro-datasets`, make sure that the package is installed in your current environment. You can do so by running `pip install kedro-datasets` or `pip
install kedro-datasets[<dataset-group>]` to install `kedro-datasets` along with related dependencies for the specific dataset group.

after_context_created

AzureSecretsHook

context

credentials

context.config_loader["credentials"] = {
            **context.config_loader["credentials"],
            **adls_creds,
        }
self.credentials = context.config_loader["credentials"]

AzureSecretsHook

AzureSecretsHook.get_creds()

@staticmethod
    def get_creds():
        return AzureSecretsHook.credentials

AbstractDataset

class PolarsDeltaDataset(AbstractDataset):
    def __init__(self, filepath: str, mode: str = "append"):
        self.filepath = filepath
        self.mode = mode

    def _load(self) -> pl.DataFrame:
        return pl.read_delta(self.filepath)

    def _save(self, data: pl.DataFrame) -> None:
        write_deltalake(
            self.filepath,
            data,
            mode=self.mode
        )

    def _describe(self):
        return dict(
            filepath=self.filepath,
            mode=self.mode
        )

MemoryDataset

with KedroSession.create() as session:
    context = session.load_context()
    context.catalog.get("input").save("mydata")
    session.run(pipeline_name="inference")

timestamp = datetime.utcnow().strftime("%Y-%m-%dT%H-%M-%S")

return {
    f"{timestamp}/national_ftds_ftus_ratio_df": national_ftds_ftus_ratio_df,
    f"{timestamp}/future_ftds_predictions_by_month_df": future_ftds_predictions_by_month_df,
    ...
}

forecast_outputs:
  type: partitions.PartitionedDataset
  dataset: pandas.CSVDataset
  path: s3://.../forecast/
  filename_suffix: ".csv"

PartitionedDataset

--params

parameters.yml

data_ingestion:
  queries:
    queries_folder: "${runtime_params:folder}"

conf_path = str(settings.CONF_SOURCE)
conf_loader = OmegaConfigLoader(conf_source=conf_path)
params = conf_loader["parameters"]

queries_folder = params["data_ingestion"]["queries"]["queries_folder"]

query_files = [f for f in os.listdir(queries_folder) if f.endswith(".sql")]

kedro run -p data_ingestion_s3 --params=folder=custom_folder

"folder " not found, and no default value provided.

kedro viz

kedro viz build

--conf-source

--params

PartitionedDataset

PartitionDataset

def my_node(inputs: dict[str, Callable[[], Any]]) -> dict[str, Any]:
    results = {}
    for key, value in inputs.items():
        response = my_function(value())
        results[key] = response
    return results

for