# Jobs & Pipelines: from raw tables to reports, on a schedule > Pipelines are SQL files that build warehouse tables; jobs run them, refresh dashboards and send notifications on a schedule, with a ledger of every run. Source: https://veldhost.eu/docs/pipelines-and-jobs **Jobs & Pipelines** is where your data gets worked on while you are not looking. A *pipeline* is SQL that builds tables in your [warehouse](https://veldhost.eu/docs/warehouse) from your site databases; a *job* puts pipelines, SQL scripts, dashboard refreshes and notifications in order and runs them on a schedule. Both live in one list in Manage — sidebar group "Data", next to Databases, Runs and Dashboards — and every run is on the Runs page. Included in every plan, at the budget of your warehouse tier. ## A pipeline is SQL There is no diagram to draw. A pipeline is a folder of up to **20 `.sql` files** (64 KB each) that you edit in the same CodeMirror editor as the Databases workbench, with the databases you may read completing under their names (`shop.` completes). Each file holds any number of `;`-separated statements, and there are exactly two forms: ``` CREATE OR REFRESH TABLE [warehouse.]name [KEY (col, col)] AS ; ``` - **TABLE** is materialised in the warehouse (MariaDB or PostgreSQL — whichever the target database is). With `KEY` the rows are upserted on those columns, so a nightly run only rewrites what changed; without `KEY` the table is rebuilt every run. The SELECT may read source views and other tables of the pipeline, tables of the warehouse, and any table of a site database as `database.table`. - **SOURCE VIEW** runs on the site database, in *its own* dialect, through the same read-only login as the workbench, and is streamed into the warehouse as a staging table. Put your `WHERE` there so only the rows you need travel. It may not read anything the pipeline defines. Sources are any site database, always read-only; the target is always a warehouse database. The **dependency graph and lineage are derived by the server** from the SQL: every name after `FROM` or `JOIN` that the pipeline defines is a dependency, so the build order follows from what you wrote, a loop is refused, and so is building in two warehouses. The panel under the editor shows **Problems** (as `file:line`, refreshed as you type), the **Pipeline graph** — "Source view" and "Materialised table" nodes — and **Tables**. Header buttons: **Run file** previews the open file without writing anything — the first 50 rows per table, built from 200 rows per source; **Dry run** takes the whole pipeline through a run that writes nothing; **Run pipeline** builds it for real and shows on Runs. A job can run a pipeline with *Full refresh*, which rebuilds every incremental table that once. Up to 20 pipelines per account. ## A job A job is a list of steps that run one at a time, in order. Each step may depend on earlier steps (`depends_on`: run when that step succeeded, failed, or always), and there are eight kinds: - **Run a pipeline** — with the optional full-refresh toggle; - **Run SQL** — up to 50 statements against a warehouse database, in order, stopping at the first error; - **Refresh a dashboard** — runs every panel and keeps the answers warm, so the page opens instantly; - **Send an email** — to members of this account only; always, only when the job succeeds or only when it fails; subject and body may use `{{job.name}}`, `{{run.status}}`, `{{run.duration}}`, `{{run.url}}` and `{{step..rows_written}}` (also `rows_read`, `status`); optionally attach a CSV of a query, up to 5 MB or 50,000 rows; - **Call a webhook** — one of the endpoints on your Integrations page, on failure by default; - **Wait** — up to 300 seconds; - **Run another job** — and optionally wait for it; - **Check** — a yes/no question on something an earlier step reported (rows written, rows read, status, panels warmed…) or on the first cell of one read-only SELECT, with *is*, *is not*, *is more than*, *is at least*, *is less than*, *is at most*, *is empty*, *is not empty* — and the steps after it wired to the answer. A job runs as the person who last saved it, and reaches exactly as far as they could by hand; a pipeline inside it runs as *its* last author. If that person leaves the account, the job stops until someone with access saves it again. ## Triggers and concurrency A job runs **Only when I run it** ("Runs when you press the button"), **On a schedule**, or **After another job succeeds**. Schedules come as presets — every few minutes or hours (with a from–to window and the weekdays it applies to), every hour, day, week or month — or as a custom cron expression, read in the timezone you choose. The shortest interval is **5 minutes on Pro** and **1 minute on Business**. One run at a time per account: when a scheduled run arrives while another is still going, the job either **skips the new run** (recorded as *skipped*) or **queues it**, up to three waiting. 500 runs a day per account. ## Budgets per run Every run is held to the account's ceiling — the budget of its warehouse tier; a job may lower its own budget, never raise it. On **Starter** (Warehouse XS) and **Pro** (S): 15 minutes, 500,000 rows, 512 MB extracted from sources, 512 MB of memory. On **Business**: 60 minutes, 5 million rows, 2 GB, 1 GB — its published budget, kept on top of Warehouse M. Warehouse L on Starter or Pro: 60 minutes, 5 million rows, 2 GB, 2 GB. A run that crosses its budget is stopped and says so. Runs and their logs are kept for 90 days. ## Job minutes, the allowance and your spend cap Every job and pipeline run is metered by the seconds the platform actually spent on it. Each warehouse tier includes job minutes a month — **30 on XS, 120 on S, 600 on M, 2,000 on L** — and only the minutes above that allowance are metered at **€0.02 per minute**, billed per second, as one line on your next invoice. Dashboard refreshes, editor, API and MCP queries and CSV imports are free. A run that fails for a reason on our side is never billed — the worker stopped, we could not start it; a run that failed on your own SQL is time we spent and counts. Your **spend cap** is **€10 a month** by default, **€50 on Business**, and you change it on the [billing page](https://manage.veldhost.eu/portal/billing) (0 means "my included minutes and nothing above them") — nothing above the cap is ever billed. You are told at 50, 80 and 100 % of the cap by email. At 100 % scheduled and chained runs are recorded as *skipped* with the reason *cap*, and "Run now" — in Manage, through the API or from your AI assistant — is refused with a link to raise the cap; raising it takes effect at once, and the month turning resets the meter. Before you switch a schedule on, the job page estimates "≈ N min/month from your last run" so you can see where it lands against your allowance. ## The Runs page [Runs](https://manage.veldhost.eu/portal/runs) lists "every unit of compute this account has used" — job runs, pipeline runs, dashboard refreshes, editor statements, API and MCP queries, imports — with the metered time added up for the window and, under it, this month's allowance: "X of Y minutes included this month", and, once minutes are priced, the amount above it so far and your cap. **Metered compute** is time; "Nothing here is a bill." Filter by kind, status, who launched it and dates; open a run for its *Steps*, the *Log* drawer per step or for the whole run, and *Stop* one that is still going. ## Limits The published limits are the standard offer: - 20 jobs and 20 pipelines per account, 20 steps per job, 20 files of 64 KB per pipeline; - schedules from every 5 minutes (Business and Warehouse L: every minute), one run at a time, three queued, 500 runs a day; - per run 15 min / 500,000 rows / 512 MB / 512 MB memory on Starter and Pro (Business: 60 min / 5 M rows / 2 GB / 1 GB; Warehouse L: 60 min / 5 M rows / 2 GB / 2 GB); - job minutes a month included: 30 on Starter, 120 on Pro, 600 on Business, 2,000 on Warehouse L — runs above them cost €0.02 per minute, billed per second, under a spend cap you set (€10 by default, €50 on Business; nothing billed above it); - a bigger tier from the Plans page: Warehouse M +€12/mo (included on Business), Warehouse L +€29/mo, extra disk €5/mo per 100 GB block; - runs kept 90 days; job runs are metered by compute time at €0.02 per minute above your allowance, shown on Runs before anything is billed. Runs that failed for platform reasons are never counted; dashboards and queries are free. **Need more?** [Contact us](mailto:hello@veldhost.eu) — larger limits are available on request, for a fee. ## From a script or your AI assistant Jobs and pipelines are in the [API](https://veldhost.eu/docs/api-reference) and the [MCP server](https://veldhost.eu/docs/connect-ai-assistant), behind the opt-in **`data`** scope: - `GET` / `POST /api/v1/jobs`, `GET` / `PUT` / `DELETE /api/v1/jobs/{id}`, `POST /api/v1/jobs/{id}/run`, `GET /api/v1/jobs/{id}/runs`, `GET /api/v1/jobs/runs`, `POST /api/v1/jobs/runs/{id}/cancel`; - `GET` / `POST /api/v1/pipelines`, `GET` / `PUT` / `DELETE /api/v1/pipelines/{id}`, `POST …/compile` (pure and free — problems with file and line), `POST …/preview` (writes nothing), `POST …/run`, `GET …/runs`. An assistant gets `list_jobs`, `create_job`, `run_job`, `get_job_runs`, `list_pipelines`, `create_pipeline`, `preview_pipeline` and `run_pipeline` — the whole pipeline grammar is in the `create_pipeline` tool's description, so a connected assistant can write a pipeline for you and check it compiles before saying it is done. A run answered as `skipped` or `queued` is the platform declining on purpose, not a failure.