Hello All,

I Need help any working professionals in this group for mlops engineer?

@elegant-energy-77053 this channel is good to ask about ML or MLOps questions 🙂 What are you working on?

Hello Team, As mlops engineer we should be aware on machine algorithms can you please let me know the tips and tricks machine algorithms to learn?

Hello All,

i am doing poc task for my company

cancer_project

i am using sypder and Anaconda tools along with using mlflow ui

i am facing some issue in my code can somone help me

import mlflow import mlflow.sklearn from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.tree import DecisionTreeClassifier from sklearn.metrics import classification_report # Function to log preprocessing steps def log_preprocessing(preprocessing_steps): mlflow.log_dict(preprocessing_steps, "preprocessing_steps.json") # Define a function to handle model training, evaluation, and logging def train_evaluate_log_model(model, model_name, X_train, X_test, y_train, y_test, params, preprocessing_steps): # Log preprocessing steps log_preprocessing(preprocessing_steps) # Train the model model.fit(X_train, y_train) # Predict and evaluate y_pred = model.predict(X_test) print(f"{model_name}:") print(classification_report(y_test, y_pred)) # Generate classification report as a dictionary class_report = classification_report(y_test, y_pred, output_dict=True) # Log with MLflow with mlflow.start_run(): mlflow.log_params(params) mlflow.log_metrics({ 'accuracy': class_report['accuracy'], 'recall_class_0': class_report['0']['recall'], 'recall_class_1': class_report['1']['recall'], 'f1_score': class_report['macro avg']['f1-score'] }) mlflow.sklearn.log_model(model, model_name) print(f"Model {model_name} logged successfully.\n") # Load and preprocess data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=102) # Preprocessing steps scaler = StandardScaler() X_train = scaler.fit_transform(X_train) X_test = scaler.transform(X_test) # Record preprocessing steps preprocessing_steps = { "scaling": { "method": "StandardScaler", "mean": scaler.mean_.tolist(), # mean used for scaling "var": scaler.var_.tolist(), # variance used for scaling }, "train_test_split": { "test_size": 0.2, "random_state": 102 } } # Set up MLflow experiment mlflow.set_experiment("cancer_data") mlflow.set_tracking_uri(uri="http://127.0.0.1:5000/") # Logistic Regression logistic_params = { "solver": "lbfgs", "max_iter": 10000, "multi_class": "auto", "random_state": 8888, } logistic_model = LogisticRegression(**logistic_params) train_evaluate_log_model(logistic_model, "Logistic Regression", X_train, X_test, y_train, y_test, logistic_params, preprocessing_steps) # Decision Tree Classifier dt_params = { "random_state": 8888, } dt_model = DecisionTreeClassifier(**dt_params) train_evaluate_log_model(dt_model, "Decision Tree Classifier", X_train, X_test, y_train, y_test, dt_params, preprocessing_steps) # You can add other models like SVM, RandomForest, and XGBoost similarly: # Example: # from sklearn.ensemble import RandomForestClassifier # rf_params = {"n_estimators": 100, "random_st

Error in sypder python interpreter

Traceback (most recent call last): Cell In[4], line 42 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=102) NameError: name 'X' is not defined

Can someone help me ?

fixed issue successfully

Anyone have a favorite data annotation tool for computer vision?

@acoustic-nest-94594 has left the channel

@dazzling-spring-85404 has left the channel

@ripe-nest-20732 has left the channel

@brash-ice-98462 has left the channel

Hi, I'd like to specify the amount and quantity of GPUs when running tasks in the Google Kubernetes Engine. Currently, I do that with a task specification like this:

@task(
    requests=Resources(cpu="8", mem="54Gi", gpu="2"),
    limits=Resources(cpu="100", mem="1Ti"),
    pod_template=PodTemplate(
        pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary",
                ),
            ],
            node_selector={
                "cloud.google.com/gke-accelerator": "nvidia-l4",
                "cloud.google.com/gke-accelerator-count": "2",
            },
        )
    ),
)

I see that Flyte also has a features for selecting GPUs: https://docs.flyte.org/en/latest/api/flytekit/extras.accelerators.html However, if I remove the pod_template and just add the accelerator kwarg, then the flytepropellor gives the following error:

│ E0113 12:02:55.686281       1 workers.go:103] error syncing '-': failed at Node[-]. Runt │
│ imeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [container]: [GKE Warden constraints violat │
│ ons[] failed to create resource, caused by: admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE W │
│ arden rejected the request because it violates one or more constraints.                                                                       │
│ Violations details: {"[denied by autogke-gpu-limitation]":["When requesting 'nvidia.com/gpu' resources, you must specify either node selector │
│  'cloud.google.com/gke-accelerator' with accelerator type or node selector 'cloud.google.com/compute-class' with existing custom compute clas │
│ s which has at least one GPU priority rule."]}

This suggests that the right GKE config is not properly set by providing the accelerator kwarg. Is this supposed to happen? If not, what is the point of the accelerator kwarg?

@future-boots-58005 has left the channel

Hi! I'm running Flyte on Google Kubernetes Engine and I'd like interruptible tasks to be run as GKE Spot Pods. I'm configuring

k8s:
      plugins:
        k8s:
          interruptible-node-selector:
            <http://cloud.google.com/gke-spot|cloud.google.com/gke-spot>: "true"

However, this conflicts with the default setting

default-annotations:
  <http://cluster-autoscaler.kubernetes.io/safe-to-evict|cluster-autoscaler.kubernetes.io/safe-to-evict>: "false"

While I could override this default, is there a way to set the

safe-to-evict

annotation only on all the non-interruptible tasks?

Has anyone used Flyte for hyperparameter tuning? Curious if anyone has any thoughts on good libraries to use. Katib and Ray Tune seem like popular options on Google. I also see that there's anew flytekit-optuna plugin.

Greetings I’m new to flyte, and I’m having an issue saving a file to my local directory in my codebase, when I use flyte I don’t really know where it’s saved? How to I save the file/ model output to my local directory outside flytes default container saving?

Hi I want to deploy Flyte onto my EKS cluster and create some basic workflows. Is there any tutorial for that? I did deploy helm chart and some pods did not start, they needed postgre so I am looking for something more comprehensive solution/tutorial. Please assist.

Hi all, I am basically ops (who used to be a developer) guy trying to get some reasonable workflow going for our ML team, who are basically bunch of students... Do you have any decent articles on how to properly organize a ML setup and somewhat understand how ML guys are actually working?

Does it make sense to allow Protobufs as a return type for Flyte tasks? Some places use protobuf quite extensively so it might be a lot cleaner to just return

pb2_my_proto

from functions. Happy to take a crack at a PR.

@fancy-car-5539 has left the channel

How to deploy using flyte, I need a code snippet on how it’s been done?

Hello all, I'm working on an NLP project for finding semantic textual similarity between 2 text paragraphs. So far, I have built a model that assesses the similarity value on a range of 0-1 between each pair of sentences from these 2 paragraphs. Not sure if this is the right channel to ask this question, but I'm just wondering if anyone has experience deploying the model in a cloud service provider by exposing it as a Server API endpoint for predicting the similarity score?