Functions Definition Language (FDL)

OSCAR services are typically defined via the Functions Definition Language (FDL) to be deployed via the OSCAR CLI. Alternative approaches are using the web-based wizard in the OSCAR Dashboard or, for a programmatic integration, via the OSCAR API.

ℹ️

It is called Functions Definition Language instead of Services Definition Language, because the definition was initially designed for SCAR, which supports Lambda functions.

Example:

functions:
  oscar:
  - oscar-test:
      name: plants
      memory: 2Gi
      cpu: '1.0'
      image: grycap/oscar-theano-plants
      script: plants.sh
      isolation_level: user
      visibility: restricted
      allowed_users: 
      - "62bb11b40398f7.......926@egi.eu"
      - "5e14d33ac4abc9.......463@egi.eu"
      input:
      - storage_provider: minio.default
        path: example-workflow/in
      output:
      - storage_provider: minio.default
        path: example-workflow/med
  - oscar-test:
      name: grayify
      memory: 1Gi
      cpu: '1.0'
      image: grycap/imagemagick
      script: grayify.sh
      interlink_node_name: vega-new-vk
      expose:
        min_scale: 3 
        max_scale: 7 
        port: 5000  
        cpu_threshold: 70 
        nodePort: 30500
        set_auth: true
        rewrite_target: true
        default_command: true
        health_path: "/"
      input:
      - storage_provider: minio.default
        path: example-workflow/med
      output:
      - storage_provider: minio.default
        path: example-workflow/res
      - storage_provider: onedata.my_onedata
        path: result-example-workflow
      - storage_provider: webdav.dcache
        path: example-workflow/res
  - oscar-kserve:
      name: inference
      memory: 200Mi
      cpu: '0.5'
      image: grycap/procces
      script: script.sh
      kserve:
        type: inference
        inference:
          model_format: onnx
        storage_uri: "oci://ghcr.io/grycap/kserve-yolo8n-onnx"
        min_scale: 1
        api_version: "v2"
        cpu: '1.0'
        memory: 2Gi
      log_level: CRITICAL
      input:
      - storage_provider: minio
        path: kserve-isvc-yolo8n-onnx/input
      output:
      - storage_provider: minio
        path: kserve-isvc-yolo8n-onnx/output

storage_providers:
  onedata:
    my_onedata:
      oneprovider_host: my_provider.com
      token: my_very_secret_token
      space: my_onedata_space
  webdav:
    dcache:
      hostname: my_dcache.com
      login: my_username
      password: my_password
  rucio:
    rucio:
      host: rucio_host
      auth_host: rucio_auth_host
      account: rucio_account
      rse: rucio_rse
      refresh_token: oidc_refresh_token
      oidc_audience: oidc_audience
      token_endpoint: oidc_token_endpoint

Top level parameters

Field	Description
`functions` Functions	Mandatory parameter to define a Functions Definition Language file. Note that "functions" instead of "services" has been used in order to keep compatibility with SCAR
`storage_providers` StorageProviders	Parameter to define the credentials for the storage providers to be used in the services
`clusters` map[string]Cluster	Configuration for the OSCAR clusters that can be used as service's replicas, being the key the user-defined identifier for the cluster. Optional

Functions

Field	Description
`oscar` map[string]Service array	Main object with the definition of the OSCAR services to be deployed. The components of the array are Service maps, where the key of every service is the identifier of the cluster where the service (defined as the value of the entry on the map) will be deployed.

Service

Field	Description
`name` string	The name of the service
`cluster_id` string	Identifier for the current cluster, used to specify the cluster's StorageProvider in job delegations. OSCAR-CLI sets it using the cluster_id from the FDL. Optional. (default: "")
`image` string	Docker image for the service
`vo` string	Virtual Organization (VO) in which the user creating the service is enrolled. (Required for multitenancy)
`allowed_users` string array	Array of EGI UIDs to grant specific user permissions on the service. If empty, the service is considered as accessible to all the users with access to the OSCAR cluster. (Enabled since OSCAR version v3.0.0).
`alpine` boolean	Set if the Docker image is based on Alpine. If `true`, a custom release of the faas-supervisor will be used. Optional (default: false)
`script` string	Local path to the user script to be executed inside the container created out of the service invocation
`file_stage_in` bool	Skip the download of the input files by the faas-supervisor (default: false)
`image_pull_secrets` string array	Array of Kubernetes secrets. Only needed to use private images located on private registries.
`memory` string	Memory limit for the service following the kubernetes format. Optional (default: 256Mi)
`cpu` string	CPU limit for the service following the kubernetes format. Optional (default: 0.2)
`enable_gpu` bool	Enable the use of GPU. Requires a device plugin deployed on the cluster (More info: Kubernetes device plugins). Optional (default: false)
`enable_sgx` bool	Enable the use of SGX plugin on the cluster containers. (More info: SGX plugin documentation). Optional (default: false)
`image_prefetch` bool	Enable the use of image prefetching (retrieve the container image in the nodes when creating the service). Optional (default: false)
`total_memory` string	Limit for the memory used by all the service's jobs running simultaneously. Apache YuniKorn's scheduler is required to work. Same format as Memory, but internally translated to MB (integer). Optional (default: "")
`total_cpu` string	Limit for the virtual CPUs used by all the service's jobs running simultaneously. Apache YuniKorn's scheduler is required to work. Same format as CPU, but internally translated to millicores (integer). Optional (default: "")
`ephemeral_storage_request` string	Request size for ephemeral storage following the kubernetes format. Optional (default: "")
`delegation` string	Mode of job delegation for replicas. Optional. Values: `static` (default), `random`, `load-based`, `topsis`.
`synchronous` SynchronousSettings	Struct to configure specific sync parameters. These settings are only applied on Knative ServerlessBackend. Optional.
`expose` ExposeSettings	Allows to expose the API or UI of the application run in the OSCAR service outside of the Kubernetes cluster. Optional.
`federation` Federation	Federation configuration (topology, members, delegation, rescheduler threshold). Optional.
`log_level` string	Log level for the faas-supervisor. Available levels: NOTSET, DEBUG, INFO, WARNING, ERROR and CRITICAL. Optional (default: INFO)
`input` StorageIOConfig array	Array with the input configuration for the service. Optional
`output` StorageIOConfig array	Array with the output configuration for the service. Optional
`environment` EnvVarsMap	The user-defined environment variables assigned to the service. Optional
`annotations` map[string]string	User-defined Kubernetes annotations to be set in job's definition. Optional
`labels` map[string]string	User-defined Kubernetes labels to be set in job's definition. Optional
`interlink_node_name` string	Name of the virtual kubelet node (if you are using InterLink nodes) Optional
`isolation_level` string	Select the isolation level of the MinIO buckets: `SERVICE` or `USER` (`SERVICE` by default) Optional
`visibility` string	Select the visibility level of service: `private`, `restricted` or `public` (`private` by default) Optional
`mount` MountSettings	Configuration to mount a storage provider path inside the service container. Optional.
`volume` VolumeSettings	Configuration for an OSCAR-managed persistent volume attached to the service. Optional.
`kserve` KServeSettings	Configuration to deploy the service using KServe (`InferenceService` or `LLMInferenceService`). Optional. Depends on cluster configuration.

SynchronousSettings

Field	Description
`min_scale` integer	Minimum number of active replicas (pods) for the service. Optional. (default: 0)
`max_scale` integer	Maximum number of active replicas (pods) for the service. Optional. (default: 0 (Unlimited))

Federation

Field	Description
`group_id` string	Identifier for the federation group. Optional (default: service name).
`topology` string	Federation topology: `none`, `star`, `mesh`. Optional.
`delegation` string	Mode of job delegation for federation members. Optional. Values: `static` (default), `random`, `load-based`, `topsis`.
`rescheduler_threshold` integer	Time (in seconds) that a job (with members) can be queued before delegating it. Optional.
`members` Replica array	List of federation members to delegate jobs. Optional.

ExposeSettings

Field	Description
`min_scale` integer	Minimum number of active replicas (pods) for the service. Optional. (default: 1)
`max_scale` integer	Maximum number of active replicas (pods) for the service. Optional. (default: 10 (Unlimited))
`api_port` integer	Port inside the container where the API is exposed. (value: 0 , the service will not be exposed.)
`cpu_threshold` integer	Percent of use of CPU before creating other pod (default: 80 max:100). Optional.
`nodePort` integer	Change the access method from the domain name to the public ip. Optional.
`set_auth` bool	Create credentials for the service, composed of the service name as the user and the service token as the password. (default: false). Optional.
`auth_type` string	Authentication middleware used when `set_auth` is enabled. Supported values are `basic` (default) and `forward`. `forward` is only supported for Gateway API/Traefik exposed services and delegates checks to OSCAR service authorization. Optional.
`rewrite_target` bool	It is an expose boolean in the FDL that controls how OSCAR configures the NGINX Ingress/HTTProute rewrite for exposed services. If rewrite_target: false, ingress rewrites to /$1. If rewrite_target: true, ingress rewrites to /system/services//exposed/$1 (default: false). Optional.
`default_command` bool	Select between executing the container's default command and executing the script inside the container. (default: false). Optional.
`health_path` string	Change the service readiness and liveness check path/endpoint. (default: "/"). Optional.
`probe_mode` string	Probe path mode for exposed-service pod health checks. `legacy` (default) keeps current behavior; `direct` probes only `health_path` on the container without the OSCAR ingress prefix. Optional.

KServeSettings

Field	Description
`type` string	KServe service type. Allowed values: `inference` and `llm_inference`. Required.
`storage_uri` string	Model storage URI consumed by KServe (for example `hf://...`, `oci://...`, or other KServe-compatible URIs). Required.
`inference` KServeInferenceSettings	Inference-specific configuration. Required when `type` is `inference`. It must be omitted when `type` is `llm_inference`.
`llm_inference` KServeLLMInferenceSettings	LLM inference configuration used with `llm_inference` services. Optional.
`api_version` string	Protocol version used by KServe predictors. Allowed values: `v1`, `v2`. Optional. (default: `v1`)
`min_scale` integer	Minimum number of predictor replicas. Optional. (default: `0`; for `llm_inference`, OSCAR enforces at least `1`)
`max_scale` integer	Maximum number of predictor replicas. Optional. (default: `1`). If `min_scale` is greater than `max_scale`, OSCAR sets `max_scale` equal to `min_scale`.
`cpu` string	CPU resources for the KServe workload in Kubernetes quantity format. Optional. (default: `0.2`)
`memory` string	Memory resources for the KServe workload in Kubernetes quantity format. Optional. (default: `256Mi`)
`args` string array	Command-line arguments passed to the KServe model container. Optional.
`env` map[string]string	Environment variables passed to the KServe model container. Optional.
`enable_gpu` bool	Requests one GPU for the KServe workload (`nvidia.com/gpu: 1`). Optional. (default: false)
`set_auth` bool	Enables authentication middleware for the exposed KServe route. Optional. (default: true)

KServeInferenceSettings

Field	Description
`model_format` string	Model format expected by KServe for `inference` services. Required when `type` is `inference`. Typical values include: `onnx`, `sklearn`, `xgboost`, `pytorch`, `tensorflow`, `triton`, `huggingface`.
`runtime` string	Explicit KServe ServingRuntime name to use for `inference` services. Optional.

KServeLLMInferenceSettings

Field	Description
`runtime_image` string	Runtime image for `llm_inference` services. Optional. If omitted, OSCAR uses the default vLLM-based runtime image.

MountSettings

Field	Description
`storage_provider` string	Identifier of the storage provider. Optional.
`path` string	Path to the folder that will be mounted. Optional.

VolumeSettings

Field	Description
`name` string	Logical volume name. Optional when creating a new volume from the service definition; required when mounting an existing managed volume. Names must follow Kubernetes DNS-1123 rules.
`size` string	Requested volume size using Kubernetes quantity format (for example `1Gi`). Required when the service creates a new volume.
`mount_path` string	Absolute path inside the service container where the volume is mounted. Required when volume is set.
`lifecycle_policy` string	Lifecycle behavior for service-created volumes. Allowed values are `delete` (default) and `retain`. Ignored when mounting an existing volume.

Replica

Field	Description
`type` string	Type of the replica to re-send events (can be `oscar` or `endpoint`)
`cluster_id` string	Identifier of the cluster as defined in the "clusters" FDL field. Only used if Type is `oscar`
`service_name` string	Name of the service in the replica cluster. Only used if Type is `oscar`
`url` string	URL of the endpoint to re-send events (HTTP POST). Only used if Type is `endpoint`
`ssl_verify` boolean	Parameter to enable or disable the verification of SSL certificates. Only used if Type is `endpoint`. Optional. (default: true)
`priority` integer	Priority value to define delegation priority. Highest priority is defined as 0. If a delegation fails, OSCAR will try to delegate to another replica with lower priority. Optional. (default: 0)
`headers` map[string]string	Headers to send in delegation requests. Optional

StorageIOConfig

Field	Description
`storage_provider` string	Reference to the storage provider defined in storage_providers. This string is composed by the provider's name (minio, s3, onedata, webdav, rucio) and the identifier (defined by the user), separated by a point (e.g. "minio.myidentifier")
`path` string	Path in the storage provider. In MinIO and S3 the first directory of the specified path is translated into the bucket's name (e.g. "bucket/folder/subfolder")
`suffix` string array	Array of suffixes for filtering the files to be uploaded. Only used in the `output` field. Optional
`prefix` string array	Array of prefixes for filtering the files to be uploaded. Only used in the `output` field. Optional

EnvVarsMap

Field	Description
`variables` map[string]string	Map to define the environment variables that will be available in the service container
`secrets` map[string]string	Map to define the secret environment variables that will be available in the service container

ℹ️ For federated services, secrets.refresh_token is required. OSCAR Manager stores this value in the user namespace and does not mount it into service pods. OSCAR also injects a small set of reserved environment variables in every service container:

Variable	Description
`OSCAR_SERVICE_NAME`	Service name.
`OSCAR_SERVICE_TOKEN`	Generated OSCAR service token.
`OSCAR_SERVICE_BASE_PATH`	Base exposed path, for example `/system/services/{service_name}/exposed`. It is an empty string for non-exposed services.

These variables are managed by OSCAR and are available in addition to the user-defined entries declared in environment.variables.

StorageProviders

Field	Description
`minio` map[string]MinIOProvider	Map to define the credentials for a MinIO storage provider, being the key the user-defined identifier for the provider
`s3` map[string]S3Provider	Map to define the credentials for an Amazon S3 storage provider, being the key the user-defined identifier for the provider
`onedata` map[string]OnedataProvider	Map to define the credentials for a Onedata storage provider, being the key the user-defined identifier for the provider
`webdav` map[string]WebDavProvider	Map to define the credentials for a storage provider accessible via WebDAV protocol, being the key the user-defined identifier for the provider
`rucio` map[string]RucioProvider	Map to define the credentials for a Rucio storage provider, being the key the user-defined identifier for the provider

Cluster

Field	Description
`endpoint` string	Endpoint of the OSCAR cluster API
`auth_user` string	Username to connect to the cluster (basic auth)
`auth_password` string	Password to connect to the cluster (basic auth)
`ssl_verify` boolean	Parameter to enable or disable the verification of SSL certificates

MinIOProvider

Field	Description
`endpoint` string	MinIO endpoint
`verify` bool	Verify MinIO's TLS certificates for HTTPS connections
`access_key` string	Access key of the MinIO server
`secret_key` string	Secret key of the MinIO server
`region` string	Region of the MinIO server

S3Provider

Field	Description
`access_key` string	Access key of the AWS S3 service
`secret_key` string	Secret key of the AWS S3 service
`region` string	Region of the AWS S3 service

OnedataProvider

Field	Description
`oneprovider_host` string	Endpoint of the Oneprovider
`token` string	Onedata access token
`space` string	Name of the Onedata space

WebDAVProvider

Field	Description
`hostname` string	Provider hostname. It only works with `https` protocol. Skip the protocol.
`login` string	Provider account username
`password` string	Provider account password

RucioProvider

Field	Description
`host` string	Rucio Provider hostname. It only works with `https` protocol. Skip the protocol.
`auth_host` string	Rucio authentication provider host
`account` string	Rucio account
`rse` string	Rucio rse
`refresh_token` string	OIDC refresh token
`oidc_audience` string	OIDC audience
`token_endpoint` string	OIDC issue endpoint