Ask the Post Activity: Evaluating AI Generated Content

This activity asks you to research a news topic using The Washington Post's AI tool--Ask the Post AI. This tool can provide short, conversational responses to questions by drawing from articles published by The Washington Post newsroom from 2016 to the present. It will also provide links to the articles referenced when generating an answer.

You will compare an answer generated by this AI tool to the news coverage in one of the referenced articles.

As you work through the activity, pay attention to how the resources you consult:

• show how they were created
• reflect changing technology
• match your prior knowledge and curiosity about the topic

Please review the following article, which describes how the tool was created and what you can expect when using the Ask the Post AI tool. This link will open in a new tab. https://www.washingtonpost.com/technology/2024/11/07/faq-ask-the-post-ai/. Then answer the questions below.

When researching a news topic, you may be looking for a quick overview from a trustworthy source, or you may be looking for a nuanced, in-depth view of the topic. This will often depend on what you need the information for, how much you already know about the topic, how interested you are in the topic, and even what sources you have available to you.

Step 1 - Choose a Topic

Think of a topic that might be covered in The Washington Post that you are interested in. This can be anything, but you should have enough prior knowledge about this topic to analyze the accuracy and relevance of news coverage. The Washington Post's coverage includes topics like entertainment, style, sports, and wellness as well as areas like politics, business, and technology.

Step 2 - Think of a Question

Now that you've chosen a topic, come up with a question related to this topic--this shouldn't have a quick answer, like a yes or no. For example, instead of asking who won the World Series in 2024, someone interested in baseball might ask how the pitch clock changes baseball games. This second question is still answerable with The Washington Post articles, but it requires more interpretation and evidence.

Step 3 - Try it Out

Once you have a question, try putting it into Ask the Post AI. The following link will open in a new tab: www.washingtonpost.com/ask-the-post-ai/. This product is still in an experimental phase, so you may have to try multiple questions or reword your question before the AI tool is able to give you a response.

When you have a successful question, Ask the Post AI will display a 1-2 paragraph answer with between 2 and 5 articles that were used to generate the response linked below the response. Read through the generated answer and the titles of the linked articles, and consider the following:

• Does the AI summary answer your question?
• Do the examples or evidence listed support the answer provided?
• Is the connection between the headlines of the articles referenced and the answer the AI tool generated clear, and does it make sense?
• Is the AI tool referencing recent articles? If not, are the articles from an appropriate time period for your topic?

Choose one of the articles used to generate the answer.

Read the article, keeping the AI tool's output in mind. Consider the following:

• Does the evidence in the article match the answer given by the AI tool?
• Does the original journalist frame their topic the same way?
• Is any key information from the article missing in the AI tool's output?

Remember that the AI tool compares text from multiple articles to generate an answer, so every element of the AI tool's answer will not necessarily be addressed in each article.