Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Releases

## 0.16.5 - Chatbot Updates

Image Edit

* SwarmUI: Added support for sending an image along with the prompt (image+prompt / img2img). If an image is uploaded in the chat UI, `/image {prompt}` will pass it as an init image to SwarmUI, keeping external link behavior. Backwards compatible when no image is attached. Internally, we now populate `rawInput` with `initimage`/`maskimage` data-URLs for the Swarm API.
* New command: `/image edit {prompt}` explicitly selects edit mode. Requires an uploaded image and applies a default edit noise/strength (configurable via `IMAGE_EDIT_NOISE`, default 0.35). Helpful for workflows like object additions, style changes, or masked edits when combined with a `maskimage` in future UI updates.

Markdown Rendering in Chat UI

* Full Markdown Support: Integrated `marked.js` library (v5+) with GitHub-flavored markdown for comprehensive rendering of LLM responses.
* Supported Elements: Headers (h1-h6 with borders), bold/italic text, tables (auto-width with tight padding), bullet/numbered lists (compact spacing), blockquotes, horizontal rules, inline code, code blocks, and hyperlinks.
* Live Streaming: Real-time markdown rendering during token streaming with 300ms debounced parsing for smooth display.
* Code Block Features:
- Language detection with "Copy [language]" buttons positioned at top-right of code blocks.
- Event delegation pattern ensures copy functionality persists across all interactions.
* Raw/Rendered Toggle: Blue "View Raw" button appears on hover at bottom-right of responses, allowing users to switch between rendered markdown and raw text. Toggle persists across multiple conversation turns.
* Optimized Styling: Table styling with `width: auto` and `table-layout: auto` for content-fitted display, compact list spacing (`line-height: 1.3`, `margin: 0` on list items), and consistent formatting across all markdown elements.

Static Assets & UI Improvements

* Local JavaScript Libraries: Moved to local copies of `marked.js` and `socket.io.js` for air-gapped deployment support. All static assets now served from `/static/` directory.
* Consolidated Static File Serving: Single `/static/{filename}` route handles all static assets (JS, CSS, SVG) with automatic MIME type detection.
* Custom Favicon: Added SVG favicon (blue circle with white "T") for professional browser tab appearance.
* Proper MIME Types: Added support for `.svg` files with `image/svg+xml` content type to ensure proper rendering in browsers.


## 0.16.4 - Document Generation & Security

* Chatbot - Document Generation: End-to-end document creation (PDF, DOCX, XLSX, PPTX) with download links.
Expand Down Expand Up @@ -36,7 +62,6 @@ Example Usage

> Write an algorithm in python to sort a list of number. Create a PDF to describe how it works.


## 0.16.2 - Repetition Filter Settings

* Added environment variables for repetition filter: `REPEAT_WINDOW` and `REPEAT_COUNT`.
Expand Down
10 changes: 10 additions & 0 deletions chatbot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,3 +417,13 @@ The Chatbot can use this information if you send the prompt command:
/rag records How much did we donate to charity in 2022?
/rag blog 5 List some facts about solar energy.
```

## Credits

This project uses the following open-source libraries:

- **[Prism.js](https://prismjs.com/)** - Syntax highlighting (MIT License)
- **[marked.js](https://marked.js.org/)** - Markdown parsing (MIT License)
- **[Socket.IO](https://socket.io/)** - Real-time communication (MIT License)

See [THIRD-PARTY-LICENSES.md](THIRD-PARTY-LICENSES.md) for full license details.
9 changes: 9 additions & 0 deletions chatbot/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@

The TinyLLM Chatbot docker container is available at: [jasonacox/chatbot](https://hub.docker.com/r/jasonacox/chatbot).

## 0.16.5 - Enhanced Markdown and Code Rendering

* Added syntax highlighting for code blocks using Prism.js with support for Python, JavaScript, and HTML.
* Improved streaming output rendering with proper HTML escaping to prevent code interpretation during display.
* Enhanced code block styling with dark theme background and optimized font size and line spacing.
* Fixed content jump issue when streaming text transitions to rendered markdown by adding consistent margin spacing.
* Added comprehensive third-party license attribution for all frontend and backend libraries.
* All static assets (Prism.js, marked.js, Socket.IO) now served locally to support air-gapped deployments.

## 0.16.4 - Document Generation Support

* Added comprehensive document generation functionality with support for PDF, Word, Excel, and PowerPoint formats.
Expand Down
170 changes: 170 additions & 0 deletions chatbot/THIRD-PARTY-LICENSES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Third-Party Software Licenses

This project uses the following third-party libraries:

## Frontend Libraries (JavaScript/CSS)

### Prism.js
- **License**: MIT License
- **Copyright**: Copyright (c) 2012 Lea Verou
- **Website**: https://prismjs.com/
- **Purpose**: Syntax highlighting for code blocks in the chat interface
- **Files**:
- `app/static/prism.min.js`
- `app/static/prism.min.css`
- `app/static/prism-python.min.js`
- `app/static/prism-javascript.min.js`
- `app/static/prism-markup.min.js`

### marked.js
- **License**: MIT License
- **Copyright**: Copyright (c) 2018+, MarkedJS (https://github.com/markedjs/)
- **Website**: https://marked.js.org/
- **Purpose**: Markdown parsing and rendering in the chat interface
- **Files**: `app/static/marked.min.js`

### Socket.IO
- **License**: MIT License
- **Copyright**: Copyright (c) 2014-present Automattic <dev@cloudup.com>
- **Website**: https://socket.io/
- **Purpose**: Real-time bidirectional communication between client and server
- **Files**: `app/static/socket.io.js`

## Backend Libraries (Python)

### FastAPI
- **License**: MIT License
- **Copyright**: Copyright (c) 2018 Sebastián Ramírez
- **Website**: https://fastapi.tiangolo.com/
- **Purpose**: Modern web framework for building APIs

### Uvicorn
- **License**: BSD License
- **Website**: https://www.uvicorn.org/
- **Purpose**: ASGI web server implementation

### Python-SocketIO
- **License**: MIT License
- **Website**: https://python-socketio.readthedocs.io/
- **Purpose**: Server-side Socket.IO implementation

### OpenAI Python Client
- **License**: MIT License
- **Website**: https://github.com/openai/openai-python
- **Purpose**: OpenAI API client for LLM interaction

### Beautiful Soup 4 (bs4)
- **License**: MIT License
- **Website**: https://www.crummy.com/software/BeautifulSoup/
- **Purpose**: HTML/XML parsing for web scraping

### PyPDF
- **License**: BSD License
- **Website**: https://github.com/py-pdf/pypdf
- **Purpose**: PDF file reading and text extraction

### Pillow & pillow-heif
- **License**: HPND License (Historical Permission Notice and Disclaimer)
- **Website**: https://python-pillow.org/
- **Purpose**: Image processing and HEIF format support

### Weaviate Client
- **License**: BSD-3-Clause License
- **Website**: https://weaviate.io/
- **Purpose**: Vector database client for RAG functionality

### ReportLab
- **License**: BSD License
- **Website**: https://www.reportlab.com/opensource/
- **Purpose**: PDF document generation

### python-docx
- **License**: MIT License
- **Website**: https://python-docx.readthedocs.io/
- **Purpose**: Microsoft Word document generation

### python-pptx
- **License**: MIT License
- **Website**: https://python-pptx.readthedocs.io/
- **Purpose**: Microsoft PowerPoint document generation

### openpyxl
- **License**: MIT License
- **Website**: https://openpyxl.readthedocs.io/
- **Purpose**: Excel spreadsheet generation

### pypandoc
- **License**: MIT License
- **Website**: https://github.com/bebraw/pypandoc
- **Purpose**: Document format conversion

### aiohttp
- **License**: Apache License 2.0
- **Website**: https://docs.aiohttp.org/
- **Purpose**: Async HTTP client/server framework

### Requests
- **License**: Apache License 2.0
- **Website**: https://requests.readthedocs.io/
- **Purpose**: HTTP library for API calls

### lxml
- **License**: BSD License
- **Website**: https://lxml.de/
- **Purpose**: XML and HTML processing

### Jinja2
- **License**: BSD License
- **Website**: https://jinja.palletsprojects.com/
- **Purpose**: Template engine for HTML rendering

### Pydantic
- **License**: MIT License
- **Website**: https://pydantic-docs.helpmanual.io/
- **Purpose**: Data validation using Python type annotations

### python-dotenv
- **License**: BSD License
- **Website**: https://github.com/theskumar/python-dotenv
- **Purpose**: Environment variable management

### pandas
- **License**: BSD License
- **Website**: https://pandas.pydata.org/
- **Purpose**: Data manipulation and analysis

---

## MIT License Text

```
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

## Apache License 2.0

Libraries using Apache License 2.0 (aiohttp, requests) are licensed under terms that allow free use, modification, and distribution. Full license text available at: https://www.apache.org/licenses/LICENSE-2.0

## BSD License

Libraries using BSD License (Uvicorn, PyPDF, ReportLab, Jinja2, python-dotenv, pandas, lxml, Weaviate Client) are licensed under permissive terms similar to MIT. Each has slight variations but all allow free use, modification, and distribution with attribution.

---

All third-party libraries are used in accordance with their respective licenses.
100 changes: 90 additions & 10 deletions chatbot/app/api/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

# Third-party imports
from fastapi import FastAPI, Request, File, UploadFile, Form, HTTPException
from fastapi.responses import HTMLResponse, FileResponse
from fastapi.responses import HTMLResponse, FileResponse, Response
from fastapi.templating import Jinja2Templates
from PIL import Image
import pillow_heif
Expand Down Expand Up @@ -153,10 +153,28 @@ async def lifespan(app: FastAPI): # pylint: disable=unused-argument,redefined-ou
async def index(request: Request):
return templates.TemplateResponse(request, "index.html")

# Serve static socket.io.js
@app.get("/socket.io.js")
def serve_socket_io_js():
return FileResponse("app/templates/socket.io.js", media_type="application/javascript")
# Serve favicon.ico (return 204 No Content to avoid console errors)
@app.get("/favicon.ico")
async def favicon():
# Serve the SVG favicon
return FileResponse("app/static/favicon.svg", media_type="image/svg+xml")

# Serve static files from app/static directory
@app.get("/static/{filename}")
def serve_static_files(filename: str):
file_path = f"app/static/{filename}"
if os.path.exists(file_path):
# Determine media type based on file extension
if filename.endswith('.js'):
media_type = "application/javascript"
elif filename.endswith('.css'):
media_type = "text/css"
elif filename.endswith('.svg'):
media_type = "image/svg+xml"
else:
media_type = "application/octet-stream"
return FileResponse(file_path, media_type=media_type)
raise HTTPException(status_code=404, detail="File not found")

# Display settings and stats
@app.get("/stats")
Expand Down Expand Up @@ -204,6 +222,12 @@ async def home(format: str = None):
"Image Provider (IMAGE_PROVIDER)": IMAGE_PROVIDER,
"SwarmUI Host (SWARMUI)": SWARMUI if IMAGE_PROVIDER == "swarmui" else "Not Used",
}
# Image edit config
try:
from app.core.config import IMAGE_EDIT_NOISE
data["Image Edit Default Noise (IMAGE_EDIT_NOISE)"] = IMAGE_EDIT_NOISE
except Exception:
pass
# Add supported document formats (aliases) to status page
try:
doc_gen = get_document_generator()
Expand Down Expand Up @@ -787,7 +811,7 @@ async def handle_url_prompt(session_id, p):
async def handle_command(session_id, p):
command = p[1:].split(" ")[0].lower()
if command == "":
await sio.emit('update', {'update': '[Commands: /image /intent /model /news /rag /reset /search /sessions /stock /think /version /weather /repeat]', 'voice': 'user'}, room=session_id)
await sio.emit('update', {'update': '[Commands: /image (or /image edit) /intent /model /news /rag /reset /search /sessions /stock /think /version /weather /repeat]', 'voice': 'user'}, room=session_id)
client[session_id]["prompt"] = ''
elif command == "repeat":
await handle_repeat_command(session_id, p)
Expand Down Expand Up @@ -1051,17 +1075,73 @@ async def handle_image_command(session_id, p):
if not current_image_generator:
await sio.emit('update', {'update': '[Image Generation is not enabled]', 'voice': 'user'}, room=session_id)
return
prompt = p[6:].strip()
if not prompt:
await sio.emit('update', {'update': '[Usage: /image {prompt}] - Generate image for prompt.', 'voice': 'user'}, room=session_id)
raw_args = p[6:].strip()
if not raw_args:
await sio.emit('update', {'update': '[Usage: /image {prompt} | /image edit {prompt}]', 'voice': 'user'}, room=session_id)
return
# Support explicit edit intent: "/image edit {prompt}"
is_edit = False
prompt = raw_args
parts = raw_args.split()
if parts and parts[0].lower() in ["edit", "eidt"]:
is_edit = True
prompt = ' '.join(parts[1:]).strip()
if not prompt:
await sio.emit('update', {'update': '[Usage: /image edit {prompt}] - Provide a prompt describing the edit.', 'voice': 'user'}, room=session_id)
return
await sio.emit('update', {'update': '%s [Generating Image...]' % p, 'voice': 'user'}, room=session_id)
debug(f"Image Prompt: {prompt}")
client[session_id]["visible"] = False
client[session_id]["remember"] = True
# Generate image from current_image_generator
# If the user has uploaded an image already, pass it along as an init image (image+prompt)
gen_kwargs = {}
try:
client_image_data = client[session_id].get("image_data")
except Exception: # pragma: no cover - defensive
client_image_data = None
if client_image_data:
# client_image_data is base64 without header from our upload route
# Some frontends may include data URLs already; handle both
if client_image_data.startswith("data:image/"):
gen_kwargs["init_image_data_url"] = client_image_data
# Debug: show a short preview of the image data URL to keep logs readable
try:
preview = client_image_data[:80]
debug(f"Init image (data URL) preview: {preview}... (len={len(client_image_data)})")
except Exception:
pass
else:
gen_kwargs["init_image_b64"] = client_image_data
# Debug: show a short preview of the raw base64 image payload
try:
preview = client_image_data[:80]
debug(f"Init image (base64) preview: {preview}... (len={len(client_image_data)})")
except Exception:
pass
debug("Passing init image to generator for image+prompt mode")
elif is_edit:
# Edit mode requires an init image
await sio.emit('update', {'update': '[Image Edit requires an uploaded image. Drag-and-drop an image, then run: /image edit {prompt}]', 'voice': 'user'}, room=session_id)
return

# Apply default edit noise for explicit edit flow
if is_edit:
try:
from app.core.config import IMAGE_EDIT_NOISE
gen_kwargs["init_image_noise"] = IMAGE_EDIT_NOISE
except Exception:
pass
# If using Flux base models, stronger edit often needs max creativity (1.0) or dedicated edit models
try:
model_name = getattr(current_image_generator, "model", "") or IMAGE_MODEL
if isinstance(model_name, str) and "flux" in model_name.lower():
gen_kwargs["init_image_noise"] = 1.0
await sio.emit('update', {'update': '[Note: Flux base models may ignore edits without a mask. Consider Flux Kontext/Fill or SDXL Inpainting for stronger edits.]', 'voice': 'user'}, room=session_id)
except Exception:
pass

image_encoded = await current_image_generator.generate(prompt)
image_encoded = await current_image_generator.generate(prompt, **gen_kwargs)
if image_encoded:
# Handle different response formats - OpenAI returns raw base64, SwarmUI returns data URI
if "," in image_encoded:
Expand Down
Loading
Loading