Skip to main content
This guide covers best practices for monitor configuration, webhook implementation, and performance optimization.
Encountering issues? See Troubleshoot monitors for solutions to common problems.

Choose appropriate schedules

Match schedule frequency to your data needs and budget. Each scheduled run creates a billable job.
Use CaseScheduleRationale
News monitoringDailyBalances freshness with cost
Regulatory updatesDaily or weeklyChanges occur infrequently
Market intelligenceDaily or weeklyAdjust based on update cycles
Minimum interval: 24 hours. Schedules more frequent than daily are not supported.

Schedule formats

Define schedules in natural language with explicit timezone.
  • "every day at 12 PM UTC"
  • "every day at 9 AM EST"
Common cron patterns:
ScheduleCron ExpressionValid
"every day at 12 PM UTC"0 12 * * *✅ Daily
"every Monday at 9 AM EST"0 9 * * 1✅ Weekly
"every 6 hours"0 */6 * * *❌ Below 24h minimum

Testing procedure

Test your schedule format before production use.
1

Create test monitor

Create a monitor with your desired schedule (e.g., "every day at 3 PM UTC")
2

Wait for execution

Wait for the scheduled time to pass.
3

Verify cron expression

Check the cron_expression field:
curl "https://catchall.newscatcherapi.com/catchAll/monitors/{monitor_id}/jobs" \
  -H "x-api-key: YOUR_API_KEY"
4

Create production monitor

If correct, create your production monitor. If incorrect, adjust the schedule format.

Implement robust webhooks

Configure webhook endpoints to handle notifications reliably.

Endpoint requirements

Your webhook endpoint must:
  1. Return 2xx status code within 5 seconds.
  2. Be publicly accessible (not localhost or private network).
  3. Use HTTPS (not HTTP).
  4. Handle POST requests with JSON body.

Quick implementation

Return 200 immediately and process asynchronously to avoid timeouts:
from flask import Flask, request, jsonify
import logging

app = Flask(__name__)
logging.basicConfig(level=logging.INFO)

@app.route('/catchall/webhook', methods=['POST'])
def handle_catchall_webhook():
    try:
        # Get payload
        payload = request.json
        logging.info(f"Received webhook: {payload['monitor_id']}")

        # Return 200 immediately - process async
        process_webhook_async(payload)
        return jsonify({"status": "received"}), 200

    except Exception as e:
        logging.error(f"Webhook error: {e}")
        # Return 200 even on error to avoid retries
        return jsonify({"status": "error"}), 200

def process_webhook_async(payload):
    """Queue for background processing"""
    monitor_id = payload['monitor_id']
    records_count = payload['records_count']

    if records_count > 0:
        # Your processing logic here
        save_records(payload['records'])
If webhooks aren’t firing, see Troubleshoot: Webhook not firing.

Update monitor configuration

Modify webhook settings for existing monitors without recreating them.

When to update

Update monitor webhooks when you need to:
  • Change webhook endpoint URL
  • Modify authentication headers
  • Switch HTTP method (POST/PUT)
  • Update query parameters
Schedule and reference job cannot be modified. To change the query or schedule, create a new monitor.

Update procedure

1

Prepare new webhook config

Define your updated webhook configuration with new URL, headers, or method.
2

Send update request

    from newscatcher_catchall import CatchAllApi

    client = CatchAllApi(api_key="YOUR_API_KEY")
    monitor_id = "3fec5b07-8786-46d7-9486-d43ff67eccd4"

    # Update webhook
    response = client.monitors.update_monitor(
        monitor_id=monitor_id,
        webhook={
            "url": "https://new-endpoint.com/webhook",
            "method": "POST",
            "headers": {"Authorization": "Bearer NEW_TOKEN"}
        }
    )
    print(f"Updated: {response.monitor_id}")
3

Verify update

List your monitors to confirm the webhook configuration was updated:
    curl "https://catchall.newscatcherapi.com/catchAll/monitors" \
      -H "x-api-key: YOUR_API_KEY"
Check the webhook field in the response.
Updates take effect immediately. The next scheduled execution uses the new webhook configuration.

See also