⚙️Custom Step

A custom step in a pipeline is a step that can be configured to run your own container image on a machine, based on your CPU and memory requirements. This allows you to specify the specific resources that your pipeline needs to run, and ensures that your pipeline has access to the resources it needs to run efficiently and without interruption.

Custom steps can be useful in a number of different situations. For example, if you have a step in your ML pipeline that requires a large amount of CPU or memory resources, you can use a custom step to ensure that your pipeline has access to the resources it needs to run. This can help you avoid running into issues with resource constraints, and ensure that your pipeline runs smoothly and efficiently.

In addition to specifying the resources that your pipeline needs to run, custom steps can also be configured to run your container image on a specific machine or set of machines. This can be useful if you want to ensure that your pipeline is running on the most appropriate machines for your specific workloads. You can also configure environment variables and outputs for other steps to consume.

Overall, custom steps are a powerful and flexible way to specify the resources that your pipeline needs to run, and to ensure that your pipeline has access to the resources it needs to run efficiently and without interruption.