Assessment Autofill | FAQ

• What is Assessment Autofill?
  • Assessment Autofill is an AI-enabled feature that accelerates assessment cycles by reducing the time it takes a third-party analyst to review a third party's submitted evidence for control validation. This feature uses Natural Language Processing (NLP) and small language models to extract information from submitted evidence documents (security policies, completed questionnaires, etc.), then uses Generative AI to produce an AI response and final summarization of the results.
• Is Assessment Autofill able to answer all the Assessment questions?
  • At this time it will only provide answers for at most the 209 control questions. Currently, it is unable to provide answers at the metric level.
• What are the limitations of the Assessment Autofill AI tool?

  •  The AA AI tool is powerful, but it has important limitations users should be aware of:

    • It does not verify whether a document pertains to your company, it assumes all uploaded content is relevant.
    • It does not prioritize newer documents over older ones based on date.
    • It does not weigh document types differently (e.g., SOC vs. internal policy).
    • It does not infer the purpose of technologies mentioned unless explicitly explained in context.

    For a full breakdown of these limitations and best practices, visit the Technical Explanation document. 

• How long does it take on average to process documents?
  • The time it takes to complete an autofill run greatly depends on the volume and file size of documents included for processing. If a single file is provided, it can take on average 1-2 minutes, whereas if multiple files are provided it could take upwards of 30 minutes. The greatest length of time it could take is 1 hour.
• What file types are supported for use in Assessment Autofill?
  • Currently, the only support file type is a PDF. Many file types can be exported into a PDF however, such as Excel and Word, and then uploaded.
• What file size limit is there for documents for use in Assessment Autofill?
  • Under 100 MB is supported by the platform, however text docs greater than 20 MB are difficult for the AI tool.
• What is best practice for including documents in an autofill run (ie. singularly piece-meal vs in bulk)?
  • In bulk is best so that documents are parsed comprehensively to determine the most accurate answer based on all of the provided documents. This ensures for the greatest accuracy of the AI derived answers. If users take a piece meal approach, the system is only referencing the documents included in that run and therefore cannot cross compare accuracy of prior results with current results to determine the best answer. As a result, users may see varying results taking that approach, which is not ideal.
• What happens if I delete a document following completion of an Assessment Autofill run?
  • If users are concerned about their documents persisting in their document repository after having used it for Assessment Autofill, they may delete it at any time and the Assessment Autofill results will not be impacted. However, viewing that document and linking of documents to controls will not occur as a result.
• Is my Assessment Autofill data retained within the AI tool?
  • No , after each use of Assessment Autofill, all data used in the processing pipeline is removed leaving no trace. The information upload continues to exist and maintain security outside of the pipeline like any other document uploaded to the Exchange.
  • Here is a screenshot demonstrating how we destroy logs associated with assessment autofill runs.

• How is my data securely stored?
  • Data uploaded to Assessment Autofill is encrypted both in transit and at rest. After processing, documents are removed from the Autofill pipeline but remain securely stored within the Exchange platform, following its standard high-security protocols. Before any content is processed by AWS services, sensitive information is scrubbed, and only short, relevant excerpts are retained. Importantly, AWS does not store or use this data to train its services.
• Are my documents and autofill results used to train the AI model?
  • Only anonymized autofill results are used to train the AI model, similar to how attested control data is used for predictive training. Any identifying information—such as company name or firmographics—is removed to prevent any association with your organization. While document names are retained, no sensitive or identifiable content is used.
  • Here is a screenshot demonstrating that the AI model is trained using anonymized data. 

• How do we ensure AI-generated control answers are properly vetted before submitting the questionnaire?
  • All AI-generated answers from Assessment Autofill must be reviewed and accepted by a user before they are confirmed. This ensures that each control response is validated by a human, maintaining accuracy and accountability throughout the submission process.
• Is the AI model trained on generic data?
  • No, the AI model is specifically trained on TPRM controls and is tailored to the type of data it receives. It produces an AI-derived answer along with a summary analysis highlighting supporting points from the source text. Each answer also includes document references—limited to file name, page number, and excerpt—to show exactly where the information was derived.
• If a section in my document doesn’t apply to my company’s services, will the AI answer with “N/A”?

  • Assessment Autofill does not return “N/A” as an answer. It only suggests “Yes” or “No,” or may leave a question unanswered if no relevant section is found. If your document (e.g., ISO Statement of Applicability) includes language like “This control does not apply to our services,” the AI will likely suggest “No,” recognizing that the control is not in place. If the section is missing entirely, the AI may reference a semi-relevant part of the document, which could result in an inaccurate suggestion.

    What can you do to ensure the most accurate results when encountering this scenario? If you disagree with the AI’s autofill result, we recommend manually updating the answer or leaving it blank.

• How can we be assured that AI responses do not have hallucinations or deviate from factual documentation?

  • The AI tooling behind Assessment Autofill is a document-grounded, user-controlled, and audit-ready system that prioritizes accuracy, transparency, and security. It only processes documents you explicitly provide, and every response is generated based on that documentation — no external data or guesswork is involved.

    Each AI-generated answer includes traceable references to the source material and must be reviewed and approved by the user before submission. Combined with specialized training on third-party risk frameworks and SME-verified accuracy up to 93%, this approach minimizes the risk of hallucinations and ensures alignment with factual documentation.