Essence of Grammar Needed
|
|
|
Summary and Introduction
This webpage summarizes a multi-year experiment to identify the telltale signs of technical summaries written with artificial intelligence (AI). In this work, we found that the text for a summary generated with ChatGPT (3.5, 4, and 5) has a distinct style. One important limitation is that this AI style for ChatGPT is not necessarily the same as for other AI tools such as Claude or Gemini. Moreover, the style for a technical summary is not necessarily the same as in other documents: emails, proposals, and so forth. Still, the process to identify these signs is one that you can apply to the AI tool you use and the documents you write. Recognizing these signs for the types of document you write and the AI tools that you and others use on those documents is valuable. For instance, in our sampling, ChatGPT preferred not to repeat nouns in a summary. For that reason, rather than repeating the design term "customer need," ChatGPT would sometimes substitute another noun (such as "priority") as a synonym. Because "customer need" has a specific meaning for those in technical design, such a substitution was imprecise. To overcome this weakness, authors should instruct ChatGPT in the prompt not to insert synonyms for technical terms. Experiment to Identify Signs of AI Writing Since Spring 2023, my teaching team assigned our engineering design students to use the AI tool of their choice to write a summary. This summary is for an initial design report that the students have written and received feedback on. Overwhelmingly, as their AI tool, the students chose ChatGPT. In the first phase of our work (Spring 2023 through Fall 2025), we analyzed the grammar and style of these student summaries and identified many of the telltale signs listed here. In a second phase, we did a more controlled experiment in which we had ChatGPT 5 generate summaries for ten different design reports that were well written and that we were confidence were written by traditional means. The reports were 4 - 5 pages and presented the initial phase of a design project. Common sections included identifying customer needs and generating design concepts. The prompt that we used for those summaries was as follows: Prompt: For our attached technical report, write a summary (no more than 1000 characters) to be placed in the front of the report (below the subject line and above the Introduction). This summary should be for the report’s main text (beginning with the Introduction and going through the Conclusions, but not including any appendices). Results Shown in Figure 1 is a report summary written by an instructor in a traditional way. The report itself was drafted by a team of students using traditional means. Shown in Figure 2 is a composite AI version of the summary using the same report. In the AI version, the prompt was the one mentioned earlier in this section, and the AI tool was ChatGPT 5.3. The colors in the AI version indicate telltale signs as defined in the next two sections: stylistic signs and grammatical signs. |
Stylistics Signs of AI Writing
Presented here are the stylistic signatures [1, 2] that have arisen from analyzing ten summaries. Please note that our analysis dove much deeper than commonly mentioned telltale signs of AI such as use of em-dashes and correlative conjunctions ("not only...but also").
Grammatical Signs Presented here are the grammatical signatures [1, 2] that have arisen from analyzing ten summaries.
Other Listings of Telltale Signs Several excellent articles present telltale signs of AI-generated text [3-6]. Most of these articles focus on general writing, such as in essays and emails. In general, overlap occurs between these lists and our lists. However, our focus on technical writing and in particular a summary of an engineering design report has narrowed the types of signs that we uncovered. In addition to our analysis of AI summaries developed by students, we have surveyed the students themselves on the telltale signs that they notice. These surveys have revealed two telltale signs of AI writing that match the findings of our experiment: imprecision (hallucination and exaggeration) and long words such as -ability nouns. Students also commonly cited three other signs: fluff, em-dashes, and correlative conjunctions. Please note that the citing of these last two signs might have arisen because many of the surveyed students had not been taught these tools of writing in their previous writing classes. For instance, many students do not know that the em-dash receives its name because it is the length of the letter "m" or that this piece of punctuation is simply a horizontal version of a parenthesis. |
Figure 1. Traditionally-written summary of a design report. The text of the report, which was one of the strongest in the course, was written by a team of three students, and the summary was written by an instructor of the design course.
|
Figure 2. AI summary of the design report used for the traditional summary of Figure 1. This summary is a composite of many summaries generated with ChatGPT 4. Colors denote the telltale signs of the AI writing.
|
|
Analysis
Opening sentence with subject-noun phrase: This report... The team... Proposed concepts,,, Discovery Space staff feedback... The next steps. All five sentences begin with a subject-noun phrase. Beginning each sentence the same way leads to tiresome rhythms. Active voice verbs: ...details...identified...include...favored... involve. In all five sentences, the main verb is active. Although using active voice is generally a strength in writing, this unusually high use of active voice is a sign of AI writing. Imprecision: ...the creation of a children's science exhibit. The choice of the word "creation" is imprecise. Rather than "creation of," more precise wording would have been "design concepts for ...." The reason for this change is that the report presented only design sketches of exhibits and not the actual exhibit. |
Needless Complexity: ...PA...maintainability. In these examples, the choices of an abbreviation and long noun insert needless complexity into the writing. To overcome this weak stylistic choice, authors will need to adjust their prompts.
Ending sentences with participial phrases: ...aiming... prioritizing...ensuring...meeting: Three of the five sentences end with a participle phrase. Moreover, the last sentence ends with two stacked participle phrases. Although such a high percentage of sentences ending this way is not a weakness, it is a distinct sign. Imprecision: The next steps involve developing prototypes. The phrase "developing prototypes" is imprecise because in the design process of this report, the next steps were to generate refined concepts and then select one for prototyping. Prototyping then was the third phase (not the second phase) of the project. |
|
Quiz: Test yourself on how well you can recognize summary excerpts written by ChatGPT or humans.
|
|
References
- Michael Alley, The Craft of Scientific Writing, 4th ed. (New York: Springer, 2018).
- Michael Alley, "Section 1: Grammar," Writing as an Engineer or Scientist (1997).
- Charlie Fink, "The Seven Deadly Tells of AI Writing," Forbes (25 June 2025).
- Smriti Mallapaty, "Signs of AI-generated text found in 14% of biomedical abstracts last year," Nature (2 July 2025).
- Sam Kriss, "Why Does A.I. Write Like...That?" The New York Times Magazine (3 December 2025).
- Callum Borchers, "Why AI Workers Won't Let Bots Do the Most Basic Tasks," The Wall Street Journal (25 November 2025).
* Other than the AI example in Figure 2, AI was not used to generate text for this webpage.