Writing Style of ChatGPT
|
|
|
Introduction. Several articles in popular journals [1-4] have identified telltale signs of AI writing. However, these articles have focused on essay writing and journalism. What about technical writing? This webpage analyzes the style of report summaries generated with ChatGPT 5.3. Although the summaries were generated with AI, the technical reports that were summarized had been written in traditional ways.
Methods. To provide a reference for the style of ChatGPT-generated summaries in our study, we used the styles from the human-written reports. These reports, which ChatGPT summarized, were progress reports for the initial phase of a course's design project. What distinguished this course was its coupling of a traditional design course with an engineering writing course [5]. From more than 300 reports written over a five-year period of the course, the ten reports used in the study were chosen because of their technical precision and writing quality [6]. The reports covered five different projects, one being the design of an autonomous etch-a-sketch [7]. Having an AI tool generate a summary from a trusted document, as opposed to allowing the AI tool to generate text from its own sources, reduced the likelihood of imprecision.** Research has shown that such imprecision in technical writing arises from hallucination [8], exaggeration [9], and reference listings that either do not exist or do not support the given claim [10]. In addition, the length limit that we imposed on the summaries (no more than 1000 characters) reduced the likelihood of workslop [11-12]. To determine the style of ChatGPT, our prompt for the summary did not include any details about the style. Our prompt provided the context of the writing situation, the role of the AI tool in the document, the action ("write a summary"), and the form (no more than 1000 characters). Moreover, we used a private chat window to avoid any effects from prior chats with the AI tool. While ChatGPT was free to adopt the style of the report being summarized, we did not instruct the AI tool to do so. Results. In our analysis of the style of the summaries generated by ChatGPT, we found strengths and weaknesses. The main strength of the technical summaries generated by ChatGPT 5.3 was that these summaries conveyed the most important details of the respective reports [13]. In addition, all the sentences in the summaries added value for readers. In other words, the summaries did not contain noticeable workslop. Three other strengths were that the summaries had a logical order, primarily used active-voice verbs (about 75 percent), and in general were grammatically correct. As shown in Tables 1 and 2, ChatGPT had distinct weaknesses. First, as shown in the AI-generated summaries were more difficult to read than the original reports were. This increased difficulty for reading the ChatGPT-generated summaries revealed itself in the following statistics:
|
|
|
Besides these weaknesses and strengths, we noticed that aspects of the AI-generated text were distinctive, when compared with the student writing of the reports:
Shown in Figure 1 is a report summary written in a traditional way by a course instructor. In contrast, Figure 2 presents a composite AI-generated summary for the same report summarized in Figure 1. This composite reflects the overall style found in the thirty summaries. The colors in the AI version indicate tendencies of AI-generated writing as defined by the colors on this webpage. Presented in the Appendix is an analysis of the ChatGPT-writing signatures in the composite summary of Figure 2. Conclusions. The AI-generated summaries contained the most important details of the reports, but the style was relatively complex and lacked a variety of sentence openers to make connections between sentences. Our next step is to determine what prompts, if any, could address these two weaknesses. |
Figure 1. Traditionally-written summary of a design report. The text of the report, which was one of the strongest in the course, was written by a team of three students, and the summary was written by an instructor of the design course. Note that sentence length varies significantly, from two shorter sentences (one at 12 words and another at 13 words) to two longer sentences (one at 24 words and another at 27 words). Also, sentence openers vary beyond a subject-noun opener, with two sentences beginning with a prepositional phrase, and one sentence beginning with a participial phrase.
|
Figure 2. AI summary of the design report that was used for the traditional summary of Figure 1. This summary is a composite of thirty summaries generated with ChatGPT 5.3. As defined in upcoming subsections, colors denote different telltale signs of the AI-generated text.
|
Appendix: Analysis of ChatGPT Summary
|
Presented in this Appendix is an analysis of the ChatGPT-writing signatures in the composite summary shown back Figure 2. The colors
Opening too many sentences with subject: "This report... The team... Proposed concepts,,, Discovery Space staff feedback... The next steps...." All five sentences begin with a subject-noun phrase. Beginning each sentence the same way leads to tiresome rhythms. Active voice verbs: "...details...identified...include...involve." In all five sentences, the main verb is active. Although using active voice is a strength of writing in general, this unusually high use of active voice is a signature of AI writing. Also, the four verbs listed were used unusually often by ChatGPT. Imprecision: "...the creation of a children's science exhibit." The choice of the word "creation" here is imprecise. Rather than "creation of," more precise wording would have been "design concepts for ...." The reason for this revision is that the report presented only design sketches of exhibits and not the actual exhibit (which was built weeks later). Needless Complexity: "...PA...maintainability." In these examples, the choices of an abbreviation and long noun insert needless complexity into the writing. To overcome this weak stylistic choice, authors would need to adjust their prompts. Ending sentences with participial phrases: "...aiming... prioritizing...ensuring...meeting...." Three of the five sentences end with a participle phrase. Moreover, the last sentence ends with two stacked participle phrases. Although such a high percentage of sentences ending this way is not a weakness, it is a distinct sign. Imprecision: "The next steps involve developing prototypes to refine the design." The phrase "developing prototypes" is imprecise because in the design process of this report, the next steps were to generate refined concepts and then select one for prototyping. Prototyping then was the third phase (not the second phase) of the project. Also, the word design is imprecise because at this stage, the team had only a design concept and not a design. |
References
- Editors of The Economist, "How to Spot AI Writing: Large language models like long words and em-dashes--or do they?" Culture: Paper Trails, The Economist (30 July 2026).
- Sam Kriss, "Why Does A.I. Write Like...That?" The New York Times Magazine (3 December 2025).
- Eve Fairbanks, "The Biggest Tell That Something Was Written by AI," The Atlantic (29 May 2026).
- Charlie Fink, "The Seven Deadly Tells of AI Writing," Forbes (25 June 2025).
- Michael Alley, Stephanie Cutler, J. C. Tice, "Work-in-Progress: Embedding a large writing course in engineering design—a new model to teach technical writing," 2019 ASEE National Conference (Tampa: ASEE, 16 June 2019), DOI 10.18260/1-2--33610.
- Michael Alley, The Craft of Scientific Writing, 4th ed. (New York: Springer, 2018).
- Christina's report
- Yiqiu Shen, Laura Heacock, Keith D. Hentel, Beautriu Reig, George Shih, and Linda Moy, "ChatGPT and Other Large Language Models Are Double-edged Swords," Radiology, vol. 37, no. 2 (26 January 2023).
- Sai Anirudh Athaluri, Sandeep Varma Manthema, V. S. R. Krishna Manoj Kesapranda, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duffumpudi, "Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References," Cureus (11 April 2023), DOI: 10.7759/cureus.37432.
- Smriti Mallapaty, "Signs of AI-generated text found in 14% of biomedical abstracts last year," Nature (2 July 2025).
- BetterUp, "Workslop is the new busywork, and it's costing millions," betterup.com/workslop" (Austin, Texas: BetterUp, 2026).
- Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Aliebscher, Kristina Rapuano, and Jeffrey T. Hancock, "AI-Generated "Workshop" Is Destroying Productivity," Harvard Business Review (22 September 2025).
- Michael Alley, Hala Hobrom, and Suryansh Sijwali, “Three Stylistic Weaknesses of AI Writing in Engineering Documents and Prompts to Address Those Weaknesses,” extended abstract, 2026 ASEE St. Lawrence Regional Conference (Ithaca, NY: ASEE St. Lawrence Regional Conference, 25 April 2026).
* Other than the AI example in Figure 2, AI was not used to generate text for this webpage. However, we did use ChatGPT 5.3 to assist in calculations of statistical significance.
**Colors are used in on this page to reflect different types of stylistic and grammatical signatures: imprecision, active voice, subject sentence opener, participle phrase, and needless complexity.
**Colors are used in on this page to reflect different types of stylistic and grammatical signatures: imprecision, active voice, subject sentence opener, participle phrase, and needless complexity.