The task was to make research evidence accessible
Three investment staff tracked 450 companies. Before a committee meeting, preparing a qualitative brief took two to three days of reading filings and earnings call transcripts. The financial data was already available through FactSet. Finding and comparing what management had said still required manual work.
The team had built a prototype using uploaded PDFs and the OpenAI API. It helped with individual documents, but it could not maintain the full research library or reliably compare companies across years. Some answers included quotes that did not match the cited passages.
At NocodeAssistant, we built a persistent search system in seven weeks. We kept the scope on retrieving and explaining evidence. Analysts retained responsibility for interpreting that evidence and making investment decisions. This gave us a clear product boundary and a concrete way to test the system.
Corpus scope came before model choice
The watchlist contained 450 companies. We indexed 400 with the required filing history, covering 2,200 annual reports and transcripts. Defining that coverage was a product decision. An answer can look complete even when the underlying library is missing the relevant years or companies.
The pipeline pulls filings from SEC EDGAR and updates the library weekly. It handles 10-K and 20-F annual reports, plus transcripts filed as exhibits. New records are added without duplicating existing ones. Each run records what changed and what was skipped.
A persistent library brings ongoing ingestion and maintenance work. It also gives the team a repeatable research base across sessions. Repeated document uploads would leave coverage dependent on what each person remembered to include.
Retrieval and reading need different amounts of context
We split documents into small passages of about 220 words for search. The answer stage receives larger windows of about 650 words around the matching material. Transcripts keep an analyst’s question with the corresponding answer.
Small passages make it easier to locate a specific statement. Larger windows help the model interpret its qualifications and surrounding discussion. Using only small chunks risks losing context. Using entire filings adds processing cost and makes it harder to isolate the evidence that answers the question.
The system combines keyword and semantic search. Company names, dates, and tickers require exact matches. Questions about management tone or capital discipline also need matching by meaning.
Company identity needed its own checks
The first search version could return convincing passages from the wrong company. Matching pieces of company names caused errors. For example, the ticker DIS appeared inside the word “discussed.” We changed ticker matching to use whole words.
Common words in legal names caused a separate problem. A question containing “companies” could point the search toward Lowe’s Companies. We removed shared name words from the matching rules and added checks for conflicts when new companies entered the library.
This work belongs before answer generation. A fluent model cannot repair an evidence set that was filtered to the wrong business. We also expanded abbreviations so a query containing “AI” could find passages that used “artificial intelligence.”
A valid citation does not prove the answer is supported
We used two checks. The first checks that a citation refers to a retrieved passage. The second uses another model to compare the answer with those passages and flag statements that go beyond them.
The second check caught an unsupported inference in one answer and a comparison that discussed only one of the two requested companies. We constrained the valid citations and expanded the fallback to show five raw passages when the generated answer failed its checks.
This adds another model call and more processing time. The trade-off fits a research workflow where an analyst must check a claim before using it in a memo. The source viewer makes that check part of the normal workflow.
Weak evidence produces a no-data response. Limited coverage produces a separate notice when a company has fewer than five indexed documents. Those states need different next steps: change the question, or add the missing material.
Evaluation had to change with the library
The test set grew from 13 to 83 cases covering company matching, comparisons, historical questions, filtered searches, and questions the library could not answer. Some early tests expected a fixed company list. As the library grew, correct answers started including more companies than those tests allowed.
We changed those expectations and tested tuning changes against the full suite. Across 29 combinations, retrieving fewer candidates for broad questions, 40 instead of 80, improved ranking without reducing accuracy in the tests. Combining separately useful settings made some results worse.
The useful unit of evaluation was the complete research question, including company coverage and answer support. Improving one retrieval score was insufficient evidence to keep a change.
Ownership included the data flow and running cost
The research library stays in the client’s private database. Queries and selected passages go to the model provider through the client’s API account. That boundary makes the processing path explicit and keeps the application and corpus under the client’s control.
Infrastructure and API costs were roughly $150–250 per month at the team’s usage level. The ongoing work includes keeping ingestion, company matching, and evaluation cases current as the library grows.
The result was more research capacity
At 60 days, committee research preparation took 10 minutes instead of two to three days. The team evaluated 12–15 new positions per quarter, up from 4–5, and ran more than 50 queries per week.
I would start the evaluation suite and put a limited version in front of an associate earlier in the next build. Real research questions exposed gaps in company naming, document structure, and coverage that a clean demonstration could hide.