Jobs

A job is the core unit of work in CatchAll. You submit a query, CatchAll processes it asynchronously, and you retrieve structured JSON records when the job completes.

Jobs lifecycle

Job lifecycle diagram showing three flows: Basic (initialize optional → submit → get status → pull results), Continue (continue → get status → pull results, extends a job submitted with limit), and Audit (list user jobs, returns all jobs for your API key).

Initialize job

POST /catchAll/initialize analyzes your query and returns LLM-generated suggestions for validators, enrichments, and a date range. It does not create a job or start processing. This endpoint is a preview — it shows you one example of what validators and enrichments could look like for your query. Because the suggestions are LLM-generated, they are not deterministic. If you call initialize and then submit a job without passing any validators or enrichments, the system generates them again from scratch and the results will differ from the initialize response. To use the initialize suggestions in your job, pass them explicitly in your submit request. You can use them as-is, modify them, or write your own.

curl -X POST "https://catchall.newscatcherapi.com/catchAll/initialize" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Series B funding rounds for SaaS startups",
    "context": "Focus on funding amount and company name"
  }'

{
  "query": "Series B funding rounds for SaaS startups",
  "context": "Focus on funding amount and company name"
}

from newscatcher_catchall import CatchAllApi

client = CatchAllApi(api_key="YOUR_API_KEY")

suggestions = client.jobs.initialize(
    query="Series B funding rounds for SaaS startups",
    context="Focus on funding amount and company name"
)
print(suggestions)

import { CatchAllApiClient } from "newscatcher-catchall-sdk";

const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY" });

const suggestions = await client.jobs.initialize({
  query: "Series B funding rounds for SaaS startups",
  context: "Focus on funding amount and company name",
});
console.log(suggestions);

import com.newscatcher.catchall.CatchAllApi;
import com.newscatcher.catchall.resources.jobs.requests.InitializeRequestDto;

CatchAllApi client = CatchAllApi.builder()
    .apiKey("YOUR_API_KEY")
    .build();

var suggestions = client.jobs().initialize(
    InitializeRequestDto.builder()
        .query("Series B funding rounds for SaaS startups")
        .context("Focus on funding amount and company name")
        .build()
);
System.out.println(suggestions);

Show Initialize job response

{
  "validators": [
    {
      "name": "is_series_b_funding",
      "description": "true if the article describes a Series B funding round",
      "type": "boolean"
    },
    {
      "name": "is_saas_startup",
      "description": "true if the company receiving funding is a SaaS startup",
      "type": "boolean"
    }
  ],
  "enrichments": [
    {
      "name": "funding_amount",
      "description": "Extract the amount of funding received in the Series B round",
      "type": "number"
    },
    {
      "name": "investee_company",
      "description": "Extract the name of the SaaS startup receiving the funding",
      "type": "company"
    },
    {
      "name": "investor_company",
      "description": "Extract the name of the investor or lead investor in the funding round",
      "type": "company"
    },
    {
      "name": "funding_date",
      "description": "Extract the date when the funding round was announced",
      "type": "date"
    }
  ],
  "start_date": "2026-02-19T13:56:37.254913+00:00",
  "end_date": "2026-02-24T13:56:37.254913+00:00",
  "date_modification_message": [
    "No dates were provided; using a default window of 5 days (2026-02-19 13:56:37 to 2026-02-24 13:56:37)."
  ]
}

Create job

POST /catchAll/submit creates the job and starts processing. Only query is required — all other parameters are optional. Validators and enrichments are independent. You can define one without the other — for example, provide custom enrichments and let the system generate validators, or define validators and let the system generate enrichments. If you omit both, the system generates both from scratch based on your query. A validator applies its criteria literally to the evidence in the record’s citations. A record whose citations don’t support the criteria is dropped, rather than passed through as a borderline match. Write validator descriptions as checkable statements about the source text — if you need a broader net, loosen the description instead of relying on ambiguous cases passing.

Use limit: 10 for quick testing. If the results look good, use Continue job to process more records without restarting.

To activate Company search mode and filter results to specific companies, add connected_dataset_ids to your submit request. See Company Monitors for setup instructions.

To restrict fetching to a curated set of trusted domains, add source_groups with the slugs of the groups you want. See Source groups.

curl -X POST "https://catchall.newscatcherapi.com/catchAll/submit" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Series B funding rounds for SaaS startups",
    "context": "Focus on funding amount and company name",
    "start_date": "2026-02-18T00:00:00Z",
    "end_date": "2026-02-23T00:00:00Z",
    "limit": 10
  }'

{
  "query": "Series B funding rounds for SaaS startups",
  "context": "Focus on funding amount and company name",
  "start_date": "2026-02-18T00:00:00Z",
  "end_date": "2026-02-23T00:00:00Z",
  "limit": 10
}

job = client.jobs.create_job(
    query="Series B funding rounds for SaaS startups",
    context="Focus on funding amount and company name",
    start_date="2026-02-18T00:00:00Z",
    end_date="2026-02-23T00:00:00Z",
    limit=10
)
job_id = job.job_id
print(f"Job created: {job_id}")

const job = await client.jobs.createJob({
  query: "Series B funding rounds for SaaS startups",
  context: "Focus on funding amount and company name",
  start_date: "2026-02-18T00:00:00Z",
  end_date: "2026-02-23T00:00:00Z",
  limit: 10,
});
const jobId = job.job_id;
console.log(`Job created: ${jobId}`);

import com.newscatcher.catchall.resources.jobs.requests.SubmitRequestDto;

var job = client.jobs().createJob(
    SubmitRequestDto.builder()
        .query("Series B funding rounds for SaaS startups")
        .context("Focus on funding amount and company name")
        .startDate("2026-02-18T00:00:00Z")
        .endDate("2026-02-23T00:00:00Z")
        .limit(10)
        .build()
);
String jobId = job.getJobId();
System.out.println("Job created: " + jobId);

Show Create job response

{
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c"
}

Get job status

GET /catchAll/status/{job_id} returns the current processing stage. Poll every 30-60 seconds until status is completed. Jobs typically take 10-15 minutes. Jobs progress through these stages in order:

Stage	Description
`submitted`	Job queued, waiting to start
`analyzing`	Generating search queries, validators, and enrichments
`fetching`	Retrieving web pages
`clustering`	Grouping related web pages into event clusters
`enriching`	Validating clusters and extracting structured data
`completed`	Processing finished, results ready
`failed`	Processing failed

curl "https://catchall.newscatcherapi.com/catchAll/status/5f0c9087-85cb-4917-b3c7-e5a5eff73a0c" \
  -H "x-api-key: YOUR_API_KEY"

import time

POLL_INTERVAL_SECONDS = 60

while True:
    status = client.jobs.get_job_status(job_id)
    print(f"Status: {status.status}")

    if status.status == "completed":
        break
    elif status.status == "failed":
        print("Job failed")
        break

    time.sleep(POLL_INTERVAL_SECONDS)

const POLL_INTERVAL_MS = 60000;

while (true) {
  const status = await client.jobs.getJobStatus({ job_id: jobId });
  console.log(`Status: ${status.status}`);

  if (status.status === "completed") {
    break;
  } else if (status.status === "failed") {
    console.log("Job failed");
    break;
  }

  await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS));
}

final int POLL_INTERVAL_MS = 60000;

while (true) {
    var status = client.jobs().getJobStatus(jobId);
    System.out.println("Status: " + status.getStatus());

    if (PublicJobStatus.COMPLETED.equals(status.getStatus().orElse(null))) {
        break;
    } else if (PublicJobStatus.FAILED.equals(status.getStatus().orElse(null))) {
        System.out.println("Job failed");
        break;
    }

    Thread.sleep(POLL_INTERVAL_MS);
}

Show Get job status response

{
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
  "status": "completed",
  "steps": [
    {
      "status": "submitted",
      "order": 1,
      "completed": true
    },
    {
      "status": "analyzing",
      "order": 2,
      "completed": true
    },
    {
      "status": "fetching",
      "order": 3,
      "completed": true
    },
    {
      "status": "clustering",
      "order": 4,
      "completed": true
    },
    {
      "status": "enriching",
      "order": 5,
      "completed": true
    },
    {
      "status": "completed",
      "order": 6,
      "completed": true
    },
    {
      "status": "failed",
      "order": 7,
      "completed": false
    }
  ]
}

You can pull partial results once the job reaches the enriching stage without waiting for completion. See Get job results below.

Get job results

GET /catchAll/pull/{job_id} retrieves structured records. During the enriching stage, progress_validated tracks how many candidate clusters have been processed so far.

curl "https://catchall.newscatcherapi.com/catchAll/pull/5f0c9087-85cb-4917-b3c7-e5a5eff73a0c" \
  -H "x-api-key: YOUR_API_KEY"

# Pull after completion
results = client.jobs.get_job_results(job_id)
print(f"Found {results.valid_records} valid records")

# Or pull during enriching for early access
while True:
    status = client.jobs.get_job_status(job_id)

    if status.status in ["enriching", "completed"]:
        results = client.jobs.get_job_results(job_id)
        print(f"Progress: {results.progress_validated}/{results.candidate_records} validated, "
              f"{results.valid_records} valid")

        if status.status == "completed":
            break

    time.sleep(POLL_INTERVAL_SECONDS)

// Pull after completion
const results = await client.jobs.getJobResults({ job_id: jobId });
console.log(`Found ${results.valid_records} valid records`);

// Or pull during enriching for early access
while (true) {
  const status = await client.jobs.getJobStatus({ job_id: jobId });

  if (status.status === "enriching" || status.status === "completed") {
    const results = await client.jobs.getJobResults({ job_id: jobId });
    console.log(
      `Progress: ${results.progress_validated}/${results.candidate_records} validated, ` +
        `${results.valid_records} valid`,
    );

    if (status.status === "completed") break;
  }

  await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS));
}

// Pull after completion
var results = client.jobs().getJobResults(jobId);
System.out.println("Found " + results.getValidRecords() + " valid records");

// Or pull during enriching for early access
while (true) {
    var status = client.jobs().getJobStatus(jobId);

    if (PublicJobStatus.ENRICHING.equals(status.getStatus().orElse(null)) ||
        PublicJobStatus.COMPLETED.equals(status.getStatus().orElse(null))) {

        var results = client.jobs().getJobResults(jobId);
        System.out.println("Progress: " + results.getProgressValidated() +
            "/" + results.getCandidateRecords() + " validated, " +
            results.getValidRecords() + " valid");

        if (PublicJobStatus.COMPLETED.equals(status.getStatus().orElse(null))) break;
    }

    Thread.sleep(POLL_INTERVAL_MS);
}

Show Get job results response

{
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
  "query": "Series B funding rounds for SaaS startups",
  "context": "Focus on funding amount and company name",
  "validators": [
    "is_series_b_funding",
    "is_saas_startup"
  ],
  "enrichments": [
    "enrichment_confidence",
    "funding_amount",
    "funding_currency",
    "funding_date",
    "investee_company",
    "investor_company",
    "valuation",
    "other_investors"
  ],
  "status": "completed",
  "duration": "1m",
  "candidate_records": 4,
  "valid_records": 3,
  "progress_validated": 4,
  "date_range": {
    "start_date": "2026-02-17T00:00:00Z",
    "end_date": "2026-02-24T00:00:00Z"
  },
  "page": 1,
  "page_size": 2,
  "total_pages": 2,
  "all_records": [
    {
      "record_id": "6983973854314692457",
      "record_title": "VulnCheck Raises $25M Series B Funding",
      "enrichment": {
        "enrichment_confidence": "high",
        "funding_amount": 25000000,
        "funding_currency": "USD",
        "funding_date": "2026-02-17",
        "investee_company": {
          "source_text": "VulnCheck",
          "confidence": 0.99,
          "metadata": {
            "name": "VulnCheck",
            "domain_url": "vulncheck.com",
            "domain_url_confidence": "high"
          }
        },
        "investor_company": {
          "source_text": "Sorenson Capital",
          "confidence": 0.99,
          "metadata": {
            "name": "Sorenson Capital",
            "domain_url": null,
            "domain_url_confidence": null
          }
        },
        "valuation": 25000000,
        "other_investors": "National Grid Partners, Ten Eleven Ventures, In-Q-Tel"
      },
      "citations": [
        {
          "title": "Exclusive: VulnCheck raises $25M funding to help companies patch software bugs",
          "link": "https://www.msn.com/en-us/money/other/exclusive-vulncheck-raises-25m-funding-to-help-companies-patch-software-bugs/ar-AA1WwdjW",
          "published_date": "2026-02-17T14:01:05Z"
        },
        {
          "title": "VulnCheck lands $25m to expand threat intelligence",
          "link": "https://fintech.global/2026/02/18/vulncheck-lands-25m-to-expand-threat-intelligence",
          "published_date": "2026-02-18T10:34:27Z"
        }
      ]
    }
  ]
}

Fields of type company (like investee_company and investor_company in the example above) return a structured object with confidence scores for entity identification and domain resolution. See the Company enrichment data model.

Records from Company Search jobs include a connected_entities array with one entry per matched company — each with entity_id, name, ed_score (1–10), and relation. See Company search for the full output format.

Export results as CSV

To pull results in a spreadsheet-friendly format, use GET /catchAll/pull/{job_id}/csv. It returns the job’s records as a text/csv download instead of JSON.

curl "https://catchall.newscatcherapi.com/catchAll/pull/5f0c9087-85cb-4917-b3c7-e5a5eff73a0c/csv" \
  -H "x-api-key: YOUR_API_KEY" \
  -o job-results.csv

from newscatcher_catchall import CatchAllApi

client = CatchAllApi(api_key="YOUR_API_KEY")

csv_data = client.jobs.get_job_results_csv(
    job_id="5f0c9087-85cb-4917-b3c7-e5a5eff73a0c"
)
with open("job-results.csv", "wb") as f:
    f.write(csv_data)

import { CatchAllApiClient } from "newscatcher-catchall-sdk";

const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY" });

const csvData = await client.jobs.getJobResultsCsv({
  job_id: "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
});

import com.newscatcher.catchall.CatchAllApi;
import java.nio.file.*;

CatchAllApi client = CatchAllApi.builder()
    .apiKey("YOUR_API_KEY")
    .build();

byte[] csvData = client.jobs().getJobResultsCsv(
    "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c"
);
Files.write(Paths.get("job-results.csv"), csvData);

For agentic workflows and data pipelines, use the JSON endpoint instead. CSV and Excel are notorious for silent encoding issues, truncated long-text fields, and mangled special characters — problems that are hard to catch and harder to debug downstream.

Each record becomes one row. Enrichment fields are flattened into columns and citations are serialized as a JSON column. For event monitors, the equivalent endpoint is GET /catchAll/monitors/pull/{monitor_id}/csv, which exports the most recent run’s records. Connected entities from Company Search jobs are split into two JSON columns: event_associated_entities (direct actors) and mention_entities (passing references). These two columns appear only when the job used a connected entity dataset — if it did not, they are omitted from the export entirely rather than included as empty columns.

If you parse the CSV by column position, or expect a fixed header row, handle both shapes. Read columns by header name so exports from jobs without entity datasets don’t shift your field mapping.

Continue job

To process more records after reviewing the initial results of a job submitted with a limit, use POST /catchAll/continue. It extends the limit without restarting the job — all analysis, validation, and enrichment logic from the original job is preserved. Before continuing, check candidate_records and progress_validated in the pull response to estimate how much data remains. candidate_records is the total number of event clusters found, progress_validated is how many have been checked so far. The difference tells you roughly how many candidates are still unprocessed — note that hit rate is not linear, so the number of new valid records per continuation may vary.

curl -X POST "https://catchall.newscatcherapi.com/catchAll/continue" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
    "new_limit": 100
  }'

{
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
  "new_limit": 100
}

continued = client.jobs.continue_job(
    job_id=job_id,
    new_limit=100
)
print(f"Continued: {continued.previous_limit} → {continued.new_limit} records")

# Poll and pull again
while True:
    status = client.jobs.get_job_status(job_id)
    if status.status == "completed":
        break
    time.sleep(POLL_INTERVAL_SECONDS)

results = client.jobs.get_job_results(job_id)

const continued = await client.jobs.continueJob({
  job_id: jobId,
  new_limit: 100,
});
console.log(
  `Continued: ${continued.previous_limit} → ${continued.new_limit} records`,
);

// Poll and pull again
while (true) {
  const status = await client.jobs.getJobStatus({ job_id: jobId });
  if (status.status === "completed") break;
  await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS));
}

const results = await client.jobs.getJobResults({ job_id: jobId });

import com.newscatcher.catchall.resources.jobs.requests.ContinueRequestDto;

var continued = client.jobs().continueJob(
    ContinueRequestDto.builder()
        .jobId(jobId)
        .newLimit(100)
        .build()
);
System.out.println("Continued: " + continued.getPreviousLimit() + " → " + continued.getNewLimit());

// Poll and pull again
while (true) {
    var status = client.jobs().getJobStatus(jobId);
    if (PublicJobStatus.COMPLETED.equals(status.getStatus().orElse(null))) break;
    Thread.sleep(POLL_INTERVAL_MS);
}

var results = client.jobs().getJobResults(jobId);

Show Continue job response

{
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
  "previous_limit": 10,
  "new_limit": 100,
  "status": "accepted"
}

new_limit must be greater than the limit set in the original job.

List user jobs

GET /catchAll/jobs/user returns all jobs created with your API key, sorted by creation date, most recent first. Supports pagination via page and page_size, text query filtering via search, ownership filtering via ownership (all, own, shared), project filtering via project_id, and processing mode filtering via mode.

curl "https://catchall.newscatcherapi.com/catchAll/jobs/user" \
  -H "x-api-key: YOUR_API_KEY"

jobs = client.jobs.get_user_jobs()

for job in jobs:
    print(f"Job {job.job_id}: {job.query} ({job.status})")

const jobs = await client.jobs.getUserJobs();

for (const job of jobs) {
  console.log(`Job ${job.job_id}: ${job.query} (${job.status})`);
}

var jobs = client.jobs().getUserJobs();

for (var job : jobs.getJobs()) {
    System.out.println("Job " + job.getJobId() + ": " +
        job.getQuery() + " (" + job.getStatus() + ")");
}

Show List user jobs response

{
  "total": 27,
  "page": 1,
  "page_size": 2,
  "total_pages": 14,
  "jobs": [
    {
      "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
      "query": "Series B funding rounds for SaaS startups",
      "created_at": "2026-02-24T13:57:56.267572",
      "status": "completed",
      "mode": "base"
    },
    {
      "job_id": "8d618890-f9f5-4c97-af17-236136a306a7",
      "query": "Find corporate headquarters relocations in the US",
      "created_at": "2026-02-18T20:25:20.940731",
      "status": "completed",
      "mode": "lite"
    }
  ]
}

Filter by processing mode

Pass mode=base or mode=lite to return only jobs that ran in that mode — useful for separating full-pipeline runs from faster lite runs when reviewing history. Omit mode to list jobs in both modes. Each job in the response carries its own mode field.

cURL

curl "https://catchall.newscatcherapi.com/catchAll/jobs/user?mode=lite" \
  -H "x-api-key: YOUR_API_KEY"

Delete a job

DELETE /catchAll/jobs/{job_id} soft-deletes a job. The job is flagged as deleted and no longer appears in list results. Deletion is idempotent — deleting an already-deleted job returns 200.

curl -X DELETE "https://catchall.newscatcherapi.com/catchAll/jobs/{job_id}" \
  -H "x-api-key: YOUR_API_KEY"

client.jobs.delete_job(job_id="5f0c9087-85cb-4917-b3c7-e5a5eff73a0c")

await client.jobs.deleteJob({
  job_id: "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c",
});

client.jobs().deleteJob("5f0c9087-85cb-4917-b3c7-e5a5eff73a0c");

Response:

{
  "success": true,
  "message": "Job deleted successfully.",
  "job_id": "5f0c9087-85cb-4917-b3c7-e5a5eff73a0c"
}

Get started

Concepts

How to

API Reference

Integrations

Libraries

Jobs lifecycle

Initialize job

Create job

Get job status

Get job results

Export results as CSV

Continue job

List user jobs

Filter by processing mode

Delete a job

See also

​Jobs lifecycle

​Initialize job

​Create job

​Get job status

​Get job results

​Export results as CSV

​Continue job

​List user jobs

​Filter by processing mode

​Delete a job

​See also

Jobs lifecycle

Initialize job

Create job

Get job status

Get job results

Export results as CSV

Continue job

List user jobs

Filter by processing mode

Delete a job

See also