The server Hooks, logs and limits

Hooks, logs and limits

What the server says of itself, and what it lets each one do how often. Each is an option of createDriveHandler, and each failure of yours is logged, not the request’s.

Hooks

What is done, what failed, how long

import pino from 'pino';
import { metrics } from '@opentelemetry/api';
import { createDriveHandler, openTelemetry } from '@rebofiles/server';

const log = pino();

createDriveHandler({
  // …
  // Each thing done: a webhook, an audit of your own, another index, space counted.
  onChange: async ({ activity, entry }) => {
    await fetch(process.env.WEBHOOK!, { method: 'POST', body: JSON.stringify({ ...activity, size: entry?.kind === 'file' ? entry.versions[0].size : 0 }) });
  },
  logger: { error: (message, detail) => log.error({ detail }, message), warn: (message, detail) => log.warn({ detail }, message) },
  onRequest: openTelemetry(metrics.getMeter('rebofiles')),
});

onChange is told each thing done, as the activity says it (created, uploaded, version, renamed, moved, trashed, restored, deleted, shared, commented, held…), with the entry as it is after, null once deleted.
logger is where the server writes what it did not answer for: an activity not kept, a hook that failed, a file that could not be inspected. console unless given.
onRequest is told each route as the list says it (GET /entries/:id), its status, its time and who asked. openTelemetry(meter) makes it the http.server.request.duration histogram and a counter of the 5xx.

Limits

How much each may ask

Kind	By	Unless said
uploads	person	1000 a minute
zip	person, and link	30 a minute
searchText	person	120 a minute
linkPassword	link and address	10 wrong in 15 minutes, then not even the right one
links	link	600 a minute
requests	person	none

import { createDriveHandler } from '@rebofiles/server';

createDriveHandler({
  // …
  limits: {
    uploads: { max: 200, every: 60_000 },     // starting an upload, by person
    requests: { max: 1200, every: 60_000 },   // every request, by person
  },
  // Wrong passwords of a link counted by the address too, behind a proxy you trust.
  clientIp: (request) => request.headers.get('x-forwarded-for')?.split(',')[0].trim(),
});

Past one, the answer is a 429 rate-limited with Retry-After, and the drive says so in each person’s language. limits: false for none.

HTTP

Webhooks

Each change told to another service over HTTP, as onChange tells your code: never holding up the request that made it.

import { everyChange, verifyWebhook, webhook } from '@rebofiles/server';

createDriveHandler({
  ...options,
  // Each change POSTed as JSON, signed, tried again three times: only these actions, or all.
  onChange: everyChange(
    webhook({ url: 'https://crm.example/hooks/files', secret: process.env.HOOK_SECRET!, actions: ['uploaded', 'version', 'deleted'] }),
    ({ activity }) => audit(activity),
  ),
});

// Where it arrives: the body as it came, checked before it is trusted.
const ok = verifyWebhook(process.env.HOOK_SECRET!, await request.text(), request.headers);

The body: id (the same in each try), action, at, by, the entry in short, and the activity. Headers X-Rebofiles-Event, X-Rebofiles-Delivery, X-Rebofiles-Timestamp and X-Rebofiles-Signature, an HMAC-SHA256 of the timestamp and the body.
Tried again after a second and after four, but not when refused for what it is (4xx); onFailure is told of one that did not arrive.

Redis

More than one process

The counts are kept in the process unless a `limiter` of yours keeps them for all: a few lines over Redis.

limiter.ts

import { Redis } from 'ioredis';
import type { Limiter } from '@rebofiles/server';

const redis = new Redis(process.env.REDIS_URL!);

/** The same windows for every process: a count per key, gone when its window is. */
export const redisLimiter: Limiter = {
  async hit(key, limit, how) {
    if (how?.peek) {
      const [count, ttl] = await Promise.all([redis.get(key), redis.pttl(key)]);
      return Number(count) >= limit.max ? { ok: false, retryAfter: ttl } : { ok: true, retryAfter: 0 };
    }
    const count = await redis.incr(key);
    if (count === 1) await redis.pexpire(key, limit.every);
    return count <= limit.max ? { ok: true, retryAfter: 0 } : { ok: false, retryAfter: await redis.pttl(key) };
  },
};