Writing Style of ChatGPT
|
|
|
Introduction
Three recent articles in popular publications (The New York Times Magazine [3], The Atlantic [4], and Forbes [5]) presented lists of telltale signs of AI-generated text. All three lists included the em-dash, the sentence construction "It is not X, but Y," overusing certain words (such as the verb delve), and an unusually high percentage of lists having three items. All three of these articles, however, focused on general writing, such as essays and narratives. What about technical writing? A few articles have focused on AI-generated imprecision in technical writing. For instance, a literature review article in 2023 by Shen et al. [6] presented several examples of hallucination by ChatGPT in scientific writing. Also, an article by Athaluri et al. [7] focused on imprecise choices by ChatGPT of reference listings in scientific articles. Moreover, a 2025 article in Nature [8] found that significantly higher percentages of abstracts contained exaggeration, which is a telltale sign of AI-generated writing. In the wider category of professional writing, Niederhoffer et al. [9] discussed workslop. Rather than focusing on imprecision or workslop, our paper analyzes the grammatical and stylistic signs of AI writing. In doing so, our paper focuses on one writing situation (a technical summary of a trusted report) that is less prone to those issues. In addition, our paper focuses on one AI tool: ChatGPT. For this analysis, one important limitation is that this writing style identified for ChatGPT is not necessarily the same as what occurs for other AI tools such as Claude or Gemini. Moreover, the style that ChatGPT generates for technical summaries is not necessarily the same style as ChatGPT would generate for other types of technical documents, such as instructions or proposals. One reason for technical summaries generated by AI having a style distinct from other documents is the length limitation. In our case, that limit was no more than 1000 characters which is about 150 words.** Because this length was between 5 and 10 percent of the length of the main text, the AI tool was asked to fit the most important details of the report into a relatively tight window. For that reason, logic states that the opportunity for the AI tool to generate workslop is less for this situation than would occur in a document with an open-ended length. In addition, logic states that because the AI tool gathered all of its details from a trusted source (the main text of the vetted reports), the opportunity for imprecision (particularly, hallucination and exaggeration) should be less than for documents in the which the AI tool has to search for outside sources. Despite these limitations, the process that we used to identify the grammatical and stylistic signatures of AI-generated text in report summaries is one that could be applied to any AI tool or any type of AI-generated document. Recognizing the grammatical and stylistic signs of AI-generated text is valuable. If a reviewer recognizes that the text is likely AI-generated, the reviewer can focus on AI's documented weaknesses. As mentioned, those weaknesses include technical imprecision [6], not properly crediting sources of information [7], and a propensity for workslop [9]. In our analysis of summaries created with ChatGPT 4, for example, we found that ChatGPT 4 often would not repeat nouns in a summary [1]. As a result, rather than repeating the design term customer need, ChatGPT would substitute another noun (such as priority) as a synonym. Because customer need has a specific meaning for design engineers, such a substitution is both imprecise and confusing. To overcome this weakness, authors could instruct ChatGPT in the prompt not to insert synonyms for technical terms. Our Methods to Identify Signs of AI Writing To determine what distinguishes the style of an AI tool, we focused on one type of document (technical summaries of a trusted report) generated by a specific AI tool (ChatGPT 5.3). We selected ChatGPT because that has been the AI tool most used by our students. Each semester since Spring 2023, my teaching team asked our engineering design students to use the AI tool of their choice to write a summary of an existing design report. Over that time, more than 90 percent of the students chose ChatGPT. In our experiment, we had the latest version of ChatGPT (5.3) generate a summary for ten different design reports that were among the strongest from about 350 reports submitted over the past five years. As exemplified in one of the reports, not only were the reports well written, but after interviewing the authors, we were confident that the reports had been written by traditional means. These reports documented the initial phase of a semester-long design project. The topic for the design project varied each semester. For instance, one semester project was for an exhibit at a children's science museum and another semester project was for a line-following robot. In these initial design reports, one common section has been for identifying the customer needs, while another common section has been for generating design concepts. The prompt that we used for having ChatGPT generate those summaries was as follows: Prompt: For our attached technical report, write a summary (no more than 1000 characters**) to be placed in the front of the report (below the subject line and above the Introduction). This summary should be for the report’s main text (beginning with the Introduction and going through the Conclusions, but not including any appendices). Because we wanted to learn the writing style that the AI tool generated, our prompt did not include any specific instructions for the summary's style or grammar. However, the reports from which the AI tool generated summaries followed the writing style advocated in The Craft of Scientific Writing [9]. This book advocates three important stylistic principles at the sentence level. First, as Einstein said, "Keep things as simple as possible, yet no simpler" [10]. Second, as Theodore Bernstein, former editor of The New York Times, advocated, "One idea, one sentence" [11]. The third principle is to begin each new sentence in a way that connects with the sentence before [7]. Put another way, this third principle calls on you to begin with what is familiar before stating what is new. After selecting our AI tool and crafting a prompt, each member of our team then used the prompt to instruct ChatGPT 5.3 to summarize each of the ten selected reports. In doing so, each of us first used our personal window of ChatGPT 5.3 to generate one set of summaries. Then, we used a private window of ChatGPT to generate another set. The reason for generating these two sets was to test whether previous chats in our personal window of the chatbot had influenced the writing style of ChatGPT. For our cases, we could not detect any stylistic differences between the summaries generated in the personal and private windows. We also examined whether significant differences existed among the writing styles of the sets of summaries that each of us had ChatGPT generate. Here, we also did not detect any differences. After generating three sets of summaries, we then had a testbed of thirty summaries to analyze from grammatical and stylistic perspectives. Presented at the following page are our team's three summaries for the first report. |
Results
Shown in Figure 1 is a report summary written by an instructor in a traditional way. The report that was summarized had been written in a traditional way by a team of students. Shown in Figure 2 is a composite AI-generated summary for the same report summarized in Figure 1. This composite reflects the overall style found in the thirty summaries. The colors in the AI version indicate tendencies of AI-generated writing as defined in the next two sections: grammatical tendencies and stylistic tendencies. Presented in Appendix A is an analysis of the AI-writing signatures in the composite summary of Figure 2.
Shown in Figure 1 is a report summary written by an instructor in a traditional way. The report that was summarized had been written in a traditional way by a team of students. Shown in Figure 2 is a composite AI-generated summary for the same report summarized in Figure 1. This composite reflects the overall style found in the thirty summaries. The colors in the AI version indicate tendencies of AI-generated writing as defined in the next two sections: grammatical tendencies and stylistic tendencies. Presented in Appendix A is an analysis of the AI-writing signatures in the composite summary of Figure 2.
Figure 1. Traditionally-written summary of a design report. The text of the report, which was one of the strongest in the course, was written by a team of three students, and the summary was written by an instructor of the design course. Note that sentence length varies significantly, from two shorter sentences (one at 12 words and another at 13 words) to two longer sentences (one at 24 words and another at 27 words). Also, sentence openers vary beyond a subject-noun opener, with two sentences beginning with a prepositional phrase, and one sentence beginning with a participial phrase.
|
Figure 2. AI summary of the design report that was used for the traditional summary of Figure 1. This summary is a composite of thirty summaries generated with ChatGPT 5.3. As defined in upcoming subsections, colors denote different telltale signs of the AI-generated text.
|
|
Grammatical Tendencies of AI Writing. Presented here are the grammatical signatures [9, 12] from our analysis of the AI-generated summaries. These signatures of grammar were compared with the same grammatical aspects in the corresponding reports.
|
|
Stylistics Tendencies of AI Writing. Presented here are the stylistic signatures [9] from our analysis of the ten AI-generated summaries. These signatures of style were compared with the same aspects of style in the corresponding ten reports.
|
|
Strengths of AI Writing. For the summaries that ChatGPT 5.3 generated of the technical reports, the main strength was that the summaries communicated the most important details of the reports. In fact, the AI-generated summaries essentially had the same details as the instructor-written summaries. For earlier versions of ChatGPT (3.5 and 4), such was not the case. A second main strength of the AI summaries was that every sentence in the generated summaries counted. In other words, perhaps because of the restrictions on length, the generated summaries did not suffer from workslop. Third, the order of details was logical and relatively easy for a reader to follow. Finally, the summaries were, except for an occasional unclear pronoun reference or misplaced modifier, grammatically correct.
|
|
Weaknesses of AI Writing. For the summaries that ChatGPT 5.3 generated of the technical reports, the main weaknesses were at the word and sentence levels. Those weaknesses, which were mentioned above, are as follows:
|
|
Forthcoming: Prompt Directives to Strengthen Style of ChatGPT
|
|
Appendix A: Analysis of Composite AI Summary
Opening too many sentences with subject: "This report... The team... Proposed concepts,,, Discovery Space staff feedback... The next steps...." All five sentences begin with a subject-noun phrase. Beginning each sentence the same way leads to tiresome rhythms. Active voice verbs: "...details...identified...include...involve." In all five sentences, the main verb is active. Although using active voice is considered a strength in writing, this unusually high use of active voice is a signature of AI writing. Also, four of the verbs are often used by ChatGPT. Imprecision: "...the creation of a children's science exhibit." The choice of the word "creation" here is imprecise. Rather than "creation of," more precise wording would have been "design concepts for ...." The reason for this revision is that the report presented only design sketches of exhibits and not the actual exhibit (which was built weeks later). Needless Complexity: "...PA...maintainability." In these examples, the choices of an abbreviation and long noun insert needless complexity into the writing. To overcome this weak stylistic choice, authors would need to adjust their prompts. Ending sentences with participial phrases: "...aiming... prioritizing...ensuring...meeting...." Three of the five sentences end with a participle phrase. Moreover, the last sentence ends with two stacked participle phrases. Although such a high percentage of sentences ending this way is not a weakness, it is a distinct sign. Imprecision: "The next steps involve developing prototypes to refine the design." The phrase "developing prototypes" is imprecise because in the design process of this report, the next steps were to generate refined concepts and then select one for prototyping. Prototyping then was the third phase (not the second phase) of the project. Also, the word design is imprecise because at this stage, the team had only a design concept and not a design. |
References
- Michael Alley, Hala Hobrom, and Suryansh Sijwali, “Three Stylistic Weaknesses of AI Writing in Engineering Documents and Prompts to Address Those Weaknesses,” extended abstract, 2026 ASEE St. Lawrence Regional Conference (Ithaca, NY: ASEE St. Lawrence Regional Conference, 25 April 2026).
- Sam Kriss, "Why Does A.I. Write Like...That?" The New York Times Magazine (3 December 2025).
- Eve Fairbanks, "The Biggest Tell That Something Was Written by AI," The Atlantic (29 May 2026).
- Charlie Fink, "The Seven Deadly Tells of AI Writing," Forbes (25 June 2025).
- Yiqiu Shen, Laura Heacock, Keith D. Hentel, Beautriu Reig, George Shih, and Linda Moy, "ChatGPT and Other Large Language Models Are Double-edged Swords," Radiology, vol. 37, no. 2 (26 January 2023).
- Sai Anirudh Athaluri, Sandeep Varma Manthema, V. S. R. Krishna Manoj Kesapranda, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duffumpudi, "Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References," Cureus (11 April 2023), DOI: 10.7759/cureus.37432.
- Smriti Mallapaty, "Signs of AI-generated text found in 14% of biomedical abstracts last year," Nature (2 July 2025).
- BetterUp, "Workslop is the new busywork, and it's costing millions," betterup.com/workslop" (Austin, Texas: BetterUp, 2026).
- Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Aliebscher, Kristina Rapuano, and Jeffrey T. Hancock, "AI-Generated "Workshop" Is Destroying Productivity," Harvard Business Review (22 September 2025).
- Michael Alley, The Craft of Scientific Writing, 4th ed. (New York: Springer, 2018).
- Albert Einstein, quotation attributed by Hans Byland (1928), This quotation has a number of different forms and appeared to be an idea, which was dubbed "Einstein's razor," that Einstein revisited a number of times. One of Einstein's original forms of the idea was as follows: "Schreibe ich zu kurz, so versteht es überhaupt niemand; schreibe ich zu lang, so wird die Sache unübersichtlich."
- Theodore Bernstein, The Careful Writer (New York: Free Press, 1995).
- Michael Alley, "Section 1: Grammar," The Craft of Scientific Communication (1997).
* Other than the AI example in Figure 2, AI was not used to generate text for this webpage. However, we did use ChatGPT 5.3 to assist in calculations of statistical significance.
**As a limit on length, we used number of characters works rather number of words to eliminate word count as a possible incentive for an AI tool to select one needlessly complex word such as portability rather than the verb phrase is portable.
**As a limit on length, we used number of characters works rather number of words to eliminate word count as a possible incentive for an AI tool to select one needlessly complex word such as portability rather than the verb phrase is portable.
