\n
\u00a0Recently I’ve been working on a RAG, understanding how it works, how RAGs and Agentic technology produces tokens, and how they are trained to recognize material, and how the various layering of training accumulates information in the multi-layer massive neural network that gets trained.\u00a0 As a result of that, and my NLP background, I now have a theory about em-dashes and long sentences.<\/p>\n
When an AI trains on text, it tokenizes the content.\u00a0 The tokens it uses are typically byte-pair encodings (BPE) that are learned over a corpus; these become a new “alphabet” for language learned from character frequencies and adjacencies.\u00a0 The alphabet produced can be defined to be a fixed size or achieve a certain encoding metric over text. This results in a token alphabet of say 30,000 mappings.\u00a0 Base letters and punctuation and numbers get a BPE encoding that is an identity mapping. If the input language is expressed in the Unicode character set, and the text is UTF-8 encoded, then each byte is either a UTF-8 lower 128 character, or a lead byte for longer encodings of unicode text. BPE works on the bytes.\u00a0 Thus, any sequence of UTF-8 encoded text can be represented in the initial 256 byte pair codes that are identity mapped to bytes 0 – 255.\u00a0\u00a0Thereafter, the first most frequent pairing gets encoded as character 256.\u00a0 The text is rencoded to the new langauge left to right using the new “character”, and the process repeats.\u00a0 As a result of this process, commonly used upper 128 characters in ANSI, such as curly-quotes, mathematical symbols, en-dash, em-dash, elipses (…), et cetera, might get encoded in a single token.\u00a0 Sadly, emoji such as\u00a0\ud83c\udf89might require 4 tokens to encode.<\/p>\n
That’s the first stage.\u00a0 Then the neural network training begins.\u00a0 As that training continues, it will start to collapse concept in language that represent certain features.\u00a0 One of the features of language is the “pause”.\u00a0 The period, comma, colon, semi-colon, en-dash, and em-dash all represent pauses, and () pairs also invoke the “pause” mechanics.\u00a0 Ending of a concept or thought.\u00a0 The period plays a dual role as ending of a thought or concept but also plays a separate role (as does comma) of a digit separator and\/or decimal separator, as well as an initial separator.\u00a0 The LLM learns the idea of a conversational pause.\u00a0 Anywhere a period, comma, colon, semi-colon, en-dash or parenthetical expression appear, the em-dash can step in without further loss of meaning — making it the universal concept representing a pause.\u00a0 The LLM learns this and then decides that any pause deserves its own cluster in the model as it were.\u00a0 So all of these pauses lean towards the pause cluster.\u00a0 Since — can fit in both places the model might cluster — with , and — with ; and so forth, and when these clusters in the model again become bigger clusters of meaning, the representation of them, reversed out becomes the token having the biggest hold on the space, the –, rather than the more meaningful siblings.<\/p>\n
I believe this is an example of learned bias.\u00a0 But it also means something else.\u00a0 Thought ending behavior the period, conflates with em-dash, which is also concept ending behavior, indistinguishable from thought ending.\u00a0 And thereafter, the training loses track of the human concept of the need for a bigger pause (the period) to make concepts easier to understand.<\/p>\n
As a result of this, the training leans towards longer sentences, some taking three lines or more to complete, and therefore, you get lost in the AI noise — causing you to scratch you head and wonder “is this ever going to stop?” and then finally, it does — or not, based on the training process and random noise inputs that keep the output variable and interesting<\/p>\n
To combat this, I’ve devised a set if writing instructions (and skills because most agents I work with lose track of their instructions after a few context compressions where they haven’t been important).\u00a0 It basically says replace em-dashes with other forms of punctuation, period, comma, colon, semi-colon, or parentheses for conversational asides and clarifications.\u00a0 I have another set of skills based on HHS simple language training materials that breaks sentences down to less than N words depending on the reading score I’m reaching for.\u00a0 For technical content, about 25 words is good, 20 is better for conversational text.<\/p>\n
As I said, this is a theory, based on (what little) I know about how this stuff works, but it might be worth investigating further. The bigger pause (the blank line or newline) likely gets different treatment during learning, which is why agents might be keen on writing shorter paragraphs.\u00a0 As a theory, it’s worth investigating further, but I’m not at a point where I would be able to do that myself.<\/p>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"
\u00a0Recently I’ve been working on a RAG, understanding how it works, how RAGs and Agentic technology produces tokens, and how they are trained to recognize material, and how the various layering of training accumulates information in the multi-layer massive neural network that gets trained.\u00a0 As a result of that, and my NLP background, I now […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[96700,96697],"tags":[3038,190829,15101,241,40545],"dealstore":[],"offerexpiration":[],"class_list":["post-7027147","post","type-post","status-publish","format-standard","hentry","category-agent","category-ai","tag-agent","tag-emdashes","tag-likes","tag-long","tag-sentences"],"yoast_head":"\n
Why your AI agent likes em-dashes and long sentences - Som2ny Network<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fivemor.com\/?p=7027147\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Why your AI agent likes em-dashes and long sentences - Som2ny Network\" \/>\n<meta property=\"og:description\" content=\"\u00a0Recently I’ve been working on a RAG, understanding how it works, how RAGs and Agentic technology produces tokens, and how they are trained to recognize material, and how the various layering of training accumulates information in the multi-layer massive neural network that gets trained.\u00a0 As a result of that, and my NLP background, I now […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fivemor.com\/?p=7027147\" \/>\n<meta property=\"og:site_name\" content=\"Som2ny Network\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-04T17:18:21+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/fivemor.com\/?p=7027147#article\",\"isPartOf\":{\"@id\":\"https:\/\/fivemor.com\/?p=7027147\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371\"},\"headline\":\"Why your AI agent likes em-dashes and long sentences\",\"datePublished\":\"2026-08-04T17:18:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/fivemor.com\/?p=7027147\"},\"wordCount\":806,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/fivemor.com\/#organization\"},\"keywords\":[\"Agent\",\"emdashes\",\"Likes\",\"LONG\",\"Sentences\"],\"articleSection\":[\"Agent\",\"AI\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/fivemor.com\/?p=7027147#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/fivemor.com\/?p=7027147\",\"url\":\"https:\/\/fivemor.com\/?p=7027147\",\"name\":\"Why your AI agent likes em-dashes and long sentences - Som2ny Network\",\"isPartOf\":{\"@id\":\"https:\/\/fivemor.com\/#website\"},\"datePublished\":\"2026-08-04T17:18:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/fivemor.com\/?p=7027147#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/fivemor.com\/?p=7027147\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/fivemor.com\/?p=7027147#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/fivemor.com\/?bp_activities=1\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Why your AI agent likes em-dashes and long sentences\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/fivemor.com\/#website\",\"url\":\"https:\/\/fivemor.com\/\",\"name\":\"Som2ny Network\",\"description\":\"Daily Deals\",\"publisher\":{\"@id\":\"https:\/\/fivemor.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/fivemor.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/fivemor.com\/#organization\",\"name\":\"Som2ny Network\",\"url\":\"https:\/\/fivemor.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png\",\"contentUrl\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png\",\"width\":300,\"height\":86,\"caption\":\"Som2ny Network\"},\"image\":{\"@id\":\"https:\/\/fivemor.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/fivemor.com\"],\"url\":\"https:\/\/fivemor.com\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Why your AI agent likes em-dashes and long sentences - Som2ny Network","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fivemor.com\/?p=7027147","og_locale":"en_US","og_type":"article","og_title":"Why your AI agent likes em-dashes and long sentences - Som2ny Network","og_description":"\u00a0Recently I’ve been working on a RAG, understanding how it works, how RAGs and Agentic technology produces tokens, and how they are trained to recognize material, and how the various layering of training accumulates information in the multi-layer massive neural network that gets trained.\u00a0 As a result of that, and my NLP background, I now […]","og_url":"https:\/\/fivemor.com\/?p=7027147","og_site_name":"Som2ny Network","article_published_time":"2026-08-04T17:18:21+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/fivemor.com\/?p=7027147#article","isPartOf":{"@id":"https:\/\/fivemor.com\/?p=7027147"},"author":{"name":"admin","@id":"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371"},"headline":"Why your AI agent likes em-dashes and long sentences","datePublished":"2026-08-04T17:18:21+00:00","mainEntityOfPage":{"@id":"https:\/\/fivemor.com\/?p=7027147"},"wordCount":806,"commentCount":0,"publisher":{"@id":"https:\/\/fivemor.com\/#organization"},"keywords":["Agent","emdashes","Likes","LONG","Sentences"],"articleSection":["Agent","AI"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/fivemor.com\/?p=7027147#respond"]}]},{"@type":"WebPage","@id":"https:\/\/fivemor.com\/?p=7027147","url":"https:\/\/fivemor.com\/?p=7027147","name":"Why your AI agent likes em-dashes and long sentences - Som2ny Network","isPartOf":{"@id":"https:\/\/fivemor.com\/#website"},"datePublished":"2026-08-04T17:18:21+00:00","breadcrumb":{"@id":"https:\/\/fivemor.com\/?p=7027147#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fivemor.com\/?p=7027147"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/fivemor.com\/?p=7027147#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/fivemor.com\/?bp_activities=1"},{"@type":"ListItem","position":2,"name":"Why your AI agent likes em-dashes and long sentences"}]},{"@type":"WebSite","@id":"https:\/\/fivemor.com\/#website","url":"https:\/\/fivemor.com\/","name":"Som2ny Network","description":"Daily Deals","publisher":{"@id":"https:\/\/fivemor.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fivemor.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/fivemor.com\/#organization","name":"Som2ny Network","url":"https:\/\/fivemor.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fivemor.com\/#\/schema\/logo\/image\/","url":"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png","contentUrl":"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png","width":300,"height":86,"caption":"Som2ny Network"},"image":{"@id":"https:\/\/fivemor.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fivemor.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png","caption":"admin"},"sameAs":["https:\/\/fivemor.com"],"url":"https:\/\/fivemor.com\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts\/7027147","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7027147"}],"version-history":[{"count":0,"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts\/7027147\/revisions"}],"wp:attachment":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7027147"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7027147"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7027147"},{"taxonomy":"dealstore","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fdealstore&post=7027147"},{"taxonomy":"offerexpiration","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fofferexpiration&post=7027147"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}