Low-Code LLM Agent Platform / AgentflowV2

What the platform does

The platform lets an enterprise client’s operators build LLM chatbots and agents on a visual canvas, without writing orchestration code. Operators wire LangChain nodes into chat flows, attach knowledge bases, and publish each bot behind a prediction API that streams answers over server-sent events (SSE). A downstream conversational product calls that API, so latency and stream stability show up directly in front of end users.

It started as a fork of Flowise and grew into its own product: tenant workspaces, role-based access, monthly conversation quotas, and an English and Japanese UI. I worked on it from mid-2024 to 2026 and ended up as the top committer, then as tech lead and production release approver.

The stack:

• A Yarn and Turborepo monorepo: Express, TypeORM, and PostgreSQL on the server, React and ReactFlow on the canvas.
• BullMQ on Redis for background jobs.
• A separate Django service for authentication and Azure AD sign-in.
• Azure Container Apps, Azure AI Search, and Azure Blob Storage, provisioned with Terraform.
• Azure OpenAI, Anthropic Claude, and Google Gemini as model providers.

Streaming that the client can render

LLM tokens arrive in bursts. When the server writes each token to the SSE response the moment it arrives, the client receives clumps of text followed by silence, and a late write after the response closes throws. I put a small queue in front of the SSE writer. Writes go into the queue, a timer drains it every 50 ms, and end() resolves only after the queue is empty. The close handler clears the timer, so nothing writes to a dead socket.

A second bug hid in the lifecycle events. JSON.stringify(undefined) returns the string "undefined", which the client’s JSON.parse rejects. Normalizing empty payloads to {} fixed a class of silent stream failures. I also added first-token latency measurement per request.

Knowledge ingestion that fails one file at a time

Knowledge bases load documents, split them into chunks, embed them, and upsert them into Azure AI Search or Pinecone. The first version synced every loader with Promise.all. One failed loader rejected the whole batch, and the store’s status no longer matched what was actually indexed.

I switched to Promise.allSettled and added a status to each chunk. Only fulfilled loaders are marked synced and persisted. A failed file stays failed, and a reprocess can tell stale chunks from new ones. On the same pipeline I added embedding-model selection per knowledge base and Azure Document Intelligence parsing, with LibreOffice and CJK fonts in the image so Japanese Office files convert correctly.

Scheduled imports without stored cloud keys

The client wanted folders from their own AWS S3 bucket to sync into knowledge bases on a schedule. Storing their access keys in our database was not an option.

The design uses cross-account role assumption instead:

1. The client deploys a CloudFormation template. It creates an IAM role that trusts our AWS account, requires an ExternalId, and grants read access to one bucket prefix only.
2. At import time, the server calls STS AssumeRole with the role ARN and the ExternalId and receives short-lived credentials.
3. A BullMQ repeatable job runs the import on a cron pattern in the client’s time zone. Deleting a knowledge base removes its repeatable jobs, so no orphan schedules keep running.

1. 01 Cron fires BullMQ repeatable job in the client's time zone
2. 02 AssumeRole role ARN plus ExternalId
3. 03 Temporary credentials short-lived, never stored
4. 04 Read one prefix import, manifest, history

Each run records its history, writes a manifest of the files it imported, and maps S3 errors such as access denied or a missing key to readable messages in both languages. I prototyped the STS flow in a standalone script before touching the product code.

Shipping AgentflowV2

In 2026 we brought in AgentflowV2, Flowise’s second-generation engine: agent graphs with loops, iterations, conditions, and human-input steps. I wrote the integration plan and led the canvas, validation, tenancy, and release work. A teammate ported the server-side runtime and database changes.

Four decisions shaped the work:

• Keep old flows untouched. The prediction layer dispatches on project type, and one branch sends agent graphs to the new executor. Existing chat flows keep running on the old runtime with no change.
• Reject bad graphs before they run. Validation rules check that a loop has a positive maximum count and that its loop-back target exists upstream. They also block nested iterations and human-input steps inside an iteration, and flag duplicate condition names. These errors show on the canvas instead of failing at runtime.
• Fail closed on tenancy. Credential lists are scoped to the flow, and knowledge base lists are scoped to the flow’s workspace. When the workspace is unknown, a list returns nothing instead of everything. Sub-flow execution follows the same rule.
• Show the run as it happens. Each node streams its execution status over SSE, and the canvas renders a card per node with inputs, outputs, and tool calls.

The engine stayed behind an environment flag through UAT. We released it to production by removing the flag.

Keeping it running

Smaller pieces kept the platform healthy:

• Provider-aware token counting: tiktoken for OpenAI models, Anthropic’s token-count endpoint, and Google’s API. When chat history exceeds the model’s budget, older turns are summarized and recent turns kept.
• A fix for Chrome’s IME in the canvas editor. Writing to the store on every keystroke re-rendered the text area and broke Japanese and Vietnamese composition. The editor now keeps local state and commits on blur.
• A fix for a DNS lookup override that Node 20 broke. With autoSelectFamily on, the custom lookup has to return an array when opts.all is set, or every protected fetch fails.

I also moved UAT and production deployments from a Jenkins VM to GitHub Actions with Azure OIDC, so no cloud secrets live in CI. The build moved out of Docker: the runner builds the monorepo with caches, and the image only installs production dependencies and copies the output. I wrote the rollback runbook and the incident postmortems that went with running it in production.