Our admissions office updated its application handbook for the upcoming term. We want our Admissions Assistant to explain current application requirements and cite the new handbook.
What happened
During testing, the Agent occasionally returns language from last year’s handbook. It cites the older document, or mixes prior application deadlines with current requirements in the same response.
What I tried
I uploaded the new handbook PDF and added the updated deadline web page to the Knowledge collection. I kept the previous handbook in the collection, assuming it would serve as helpful historical background.
Environment
Agent template. Admissions Assistant.
Knowledge collection. Private office collection containing PDFs and crawled web pages.
Test data. Fictional applicant queries regarding application deadlines and required documents.
The older handbook is still reachable in the Knowledge collection and competes with the new policy file. Remove, replace, or separate superseded documents so the Agent draws on current rules only.
Why this happens
A Knowledge collection can store multiple documents covering similar topics, and adding a new handbook does not archive or invalidate the older files already in it. When current and historical documents share keywords, retrieval can return passages from both, and the model then combines them into one answer with mixed dates or outdated guidance.
What to do
Identify the active sources. Determine the exact handbook PDF and web page that represent policy for the current term.
Retire the superseded files. Remove or deactivate them in the active Knowledge collection used by your front-line Agent.
Separate the history. Place historical records you need to keep in an internal collection that no student-facing Agent uses.
Check the links. Review connected web pages so no external link points to an archived policy page showing older dates.
Name the term in the instructions. Update Agent instructions to state the active academic term and to route ambiguous policy questions to staff.
Retest. Run common admissions questions as fictional queries and confirm the responses cite only active source documents.
Two collections, two purposes:
Document
Where it belongs
Current handbook and active deadline page
The collection connected to your front-line Agent
Superseded handbooks and archived pages
A separate internal collection, unlinked from student-facing Agents
How to verify
Ask the same fictional question with different phrasing in new Chat sessions, then confirm that the Agent:
Cites the current handbook every time.
Lists active deadline dates.
Omits references to prior academic terms.
If it still does not work
Audit every file and web link connected to the collection to confirm that no older document version remains active. Work from policy documents alone, keeping applicant records and internal notes out of the collection. If the policy text itself remains ambiguous, pause Agent use for that topic until the admissions policy owner resolves it, then retest the common questions against the corrected source.