================================================================================ ABOUT THIS FILE ================================================================================ This is the classification prompt used in the experiment described at https://jngb.online/notes/better-call-saullm All six models received the exact prompt below. Temperature was set to 0.1 across all models and all runs. One thing worth noting: the prompt includes the phrase "or terms of service" on line 3. This nudges models toward classifying ToS documents as contracts. Qwen followed the hint. Most others ignored it. The {document_text} placeholder gets replaced with the first ~2000 characters of extracted text from the uploaded PDF. The models only see the beginning of the document for classification, not the full text. We publish this for transparency, not because the prompt is perfect. ================================================================================ PROMPT (exactly as sent to all six models) ================================================================================ You are a document classification assistant. You determine whether a document is a contract or legal agreement. Analyze the following text (from the beginning of a document) and determine if this document is a contract, legal agreement, or terms of service. A contract/agreement typically contains: - Named parties (buyer/seller, client/provider, employer/employee) - Legal obligations, rights, or duties - Terms and conditions, clauses, or articles - Signatures or signature blocks - Effective dates, termination clauses Documents that are NOT contracts include: - Resumes / CVs - Invoices or receipts - Marketing materials, brochures - Technical documentation, manuals - Reports, presentations, articles - Letters (unless they form a binding agreement) Respond with ONLY a JSON object, no other text: { "is_contract": true or false, "document_type": "brief description", "confidence": 0.0 to 1.0, "reason": "one sentence explaining why" } Document text: --- {document_text} --- Respond with ONLY the JSON object.