Product Engineer · Barcelona · Remote

I build AI products people actually use, and prove they work.

I'm the product owner and a core engineer for Signal, CybSafe's AI assistant. I talk to the people who use what I build, decide what's worth building, ship it across the stack, and measure it with evals. I'm also launching QuedarBien, an AI gift finder for Spain.

What I'm working on

Signal, CybSafe's AI assistant

2026 – now

Answers human-risk questions and generates security-awareness learning content for CybSafe's customers.

  • Made it cheaper and faster. Moved Signal's decision steps to faster models: about half the cost per answer and a third less latency, at near-parity quality in offline evals.
  • Lead its evaluation work. Built on Arize Phoenix with OpenTelemetry tracing: online scoring of live conversations, offline evals in CI comparing each branch against main, and LLM judges aligned against human-labelled datasets. The evals caught a model upgrade that dropped test-set answer accuracy from 93% to 33%; it shipped only once restored to 93%.
  • Made the evals cheap enough to run on every PR. Profiling showed Signal's own multi-call answering, not the judges, drove most of the cost and time. Fixing that, and parallelising judge calls (a full judge run went from 69 to 22 minutes), moved evals from occasional to CI.
  • Tested the judges themselves. Curated human-labelled datasets for the judges; refusal-detection precision rose from 52% to 98% on a 73-example tuning set, so far fewer valid answers were wrongly dropped from quality scores.
  • Built learning-module generation with the people who write them. I read the existing modules, sat with the content team to get their rules out of their heads, and turned that into a feature that builds structured modules, quizzes and images from a customer's own documents, using RAG over PostgreSQL vector storage.
  • Guardrails. A refusal judge on model output, and sanitisation of ingested context before it reaches downstream LLM calls, to limit prompt injection.
PythonDjangoOpenAIArize PhoenixOpenTelemetryLLM-as-judgeRAGpgvector

Projects

QuedarBien home page: 'El regalo que de verdad va a usar'

QuedarBien

An AI gift finder for Spain · live at quedarbien.es · launching for Christmas 2026

Most gifts end up in a drawer. Research on gift-giving says recipients value how much they use a gift, not the moment they open it. So QuedarBien asks what someone does on a normal day and what you always see them with, then finds the premium version of something they already use, at a price they'd never pay for themselves.

  • The engine: LLM steps work out the objects a person uses every day and what a real upgrade looks like, hybrid search finds candidates in a 220,000-product affiliate catalogue, and one batched scoring call ranks them on use, upgrade and how known the brand is.
  • Evaluated, not vibed: 20 realistic personas, answered the way real shoppers fill in forms (with plenty of "don't know"), scored by LLM judges each time the engine changes.
  • Run as a product: per-visitor and daily cost caps so a search costs about four cents, cookieless ad attribution, and Meta ad tests to find the hook that works.

Under the hood: FastAPI with server-rendered pages and no client JavaScript; PostgreSQL with pgvector for hybrid (keyword plus embedding) search; product feeds from Spanish retailers via affiliate networks, refreshed by a script that swaps the production table; a rate limiter keyed on a salted visitor hash so no address is stored; deployed as a Docker container on my own Coolify server.

Version one (2025): Qubi, a chat-based gift genie on LangGraph, LangChain, FastAPI and AWS with managed PostgreSQL, built to learn agentic LLM workflows hands-on. It taught me to save LLM calls for what needs them: rule-based chat gathered preferences and code-based checks replaced model calls. The rebuild keeps that lesson and drops the chat for a short form people can actually answer.

PythonFastAPIPostgreSQL + pgvectorOpenAILLM-as-judgeDockerCoolify

Elizita ↗

ELIZA (1966) rebuilt faithfully, with each descendant up to A.L.I.C.E. as a swappable rule file, and a "show workings" view explaining every reply. Built for fun.

Dermline ↗

A voice agent that answers the phone for a dermatology clinic and books appointments. I measured what makes calls feel slow, and found the fastest setups cut callers off mid-date of birth; routing turns with a local decision model fixed that. Pipecat, OpenAI, Deepgram.

Experience

Senior Engineer · CybSafe

Apr 2022 – now · Remote

Security-awareness SaaS. One of 11 engineers, kept on through several restructurings.

  • Product owner and core engineer for Signal (above).
  • Designed the event-driven trigger and workflow engine in Django and Celery, after researching established engines. It now handles about 740,000 trigger events a day and 15,000–55,000 step changes per weekday for about 145,000 enrolled users. Modelled step relationships, validation and execution state across frontend and backend.
  • Fixed scaling bottlenecks as usage grew: one Celery task per workflow instead of per user stopped large workflows flooding the queue, and bulk enrolment cut a 5,000-user benchmark from 110,005 database queries to 14, ending the timeouts.
  • Worked out how phishing simulations should behave across randomised, overlapping, multi-year campaigns, published it for customers in the help centre, then rebuilt the engine to match. The recurring bugs stopped.
  • Integrations with Slack, Microsoft Teams and Microsoft security platforms; dynamic user segmentation for campaigns.
  • Full-stack features in Shape Up cycles, including TypeScript and React.
  • Set up isolated Docker environments so several Claude Code agents can develop and QA in parallel.
DjangoCeleryPostgreSQLTypeScriptReactDockerBuildkiteSentryGrafana

Software Engineer · A Human Future

Jan 2021 – Apr 2022 · Remote
  • Built a CMS and survey builder with complex hierarchical structures in Django and HTMX.
  • Core features for a price-comparison platform: lead matching and scoring, user accounts, listings and Stripe payments.

Before software

2014 – 2021

Three years in B2B software sales and account management (Darktrace, eSight Energy, Iansyst): talking to customers, visiting their sites and working out what they needed. Then teaching English in Spain while I learned to code.

Tools

Backend
Python, Django, Django REST Framework, FastAPI, Celery, PostgreSQL (incl. pgvector)
AI
LLM evaluation (online, offline in CI, LLM-as-judge), Arize Phoenix, OpenTelemetry, RAG, guardrails, OpenAI and Anthropic APIs, Claude Code
Frontend
TypeScript, React, HTMX
Infrastructure
AWS (Cloud Practitioner), Docker, Coolify, Buildkite, Grafana, Sentry, Git
Integrations
Slack API, Microsoft Teams and Microsoft security APIs, Stripe

Contact

The best way to reach me is email: bertie.taylor.dev@gmail.com