Paper · 2024-11-19
A Layered Architecture for Developing and Enhancing Capabilities in Large Language Model-based Software Systems
Layered design for LLM-based software, with AIGNE component calls and caching as an application-layer example.
arXiv · arXiv:2411.12357v1
When an LLM-based application needs a new capability, should developers train the model, change how it generates output, or change the application around it? This 2024 paper examines that choice through a layered architecture.
It separates the system into model, inference and application layers: how the model is trained, how output is generated, and how the application organizes prompts, tools and workflows. For JSON, a data format that programs can read, the paper compares providing format examples in a prompt, constraining the format during generation, and checking and repairing output in the application. A capability may involve several layers, and the choice also depends on which components developers can modify.
Read the version record and abstract or open the paper PDF.
AIGNE in the paper
The Call Caching discussion in Section 4 uses AIGNE as an application-layer example. It describes components, including LLMs, as blocklets: software units that users or other blocklets can call. Caching interactions between them can reduce repeated nested calls.
The paper identifies a useful setting for this approach: the same request recurring within a period of time. When prompts vary between requests, these caches may be less effective.
Read the discussion in the paper, or visit the AIGNE introduction and framework documentation for later development entry points.
Paper details
| Field | Details |
|---|---|
| Authors | Dawen Zhang, Xiwei Xu, Chen Wang, Zhenchang Xing, Robert Mao |
| First submitted | November 19, 2024 |
| Version | arXiv:2411.12357v1, preprint |
| Original language | English |
| Length | 22 pages, including references |
| DOI | 10.48550/arXiv.2411.12357 |
| Original license | CC BY 4.0 |
The paper examines the framework through use cases including output formatting, creativity and call caching. Its tool and service examples belong to the 2024 research context.
Return to Publications.