[FEATURE MEGA THREAD - REQUESTS SHOULD GO HERE]

[Jakarrrg]

I have no idea how easy it is to implement, but I have seen another open-source LLM app using it (https://github.com/timmyy123/LLM-Hub) and I think that, except from the other various advantages it has by definition, it would be a very useful tool to help interconnect the different functions/tools and improve the workflow by far, making the whole app experience feel more unified.
Examples: 1) On audio scribe/voice input tools, after the result/text was given, there could be a simple button "load to global memory" (a title could be assigned as well to keep the different documents distinguished) or something like that, that would do just that. The user then could go to the chat tool and just ask for example "give me a summary of the text under the title AUDIO SCRIBE".
2) On agent skills, the possible use cases are as many as the skills are. The outcomes of most of the skills would be useful to be uploaded to global memory in order to analyse them in a conversation with the LLM model in chat tool. Translations, budget tracker data. All those could be uploaded to global memory and be readily accessible in the chat tool. I have even found a skill that can fetch text from sites (it is still a bit rough) so there the possibilities are many. As the models and skills continue to evolve it would make more sense to have this kind of functionality.

It could be something to keep in mind, if it sounds difficult right now.

[QuantSageX]

QuantSageX
opened 2 days ago · edited by QuantSageX

Dynamic Max Context Extension Using Graph RAG but all chats should be stored on device... So even if you are chatting with it ...You should in theory have infinite context length if possible it should also incorporate https://github.com/MemMachine/MemMachine ... To make the burden lighter

[jegly]

i will look into rag for 1.0.8 release. thank you

[Jakarrrg]

Another basic (in my opinion ofcourse) feature that could be implemented at some point is allowing the user to set and save multiple preconfigured system prompts that could be used for different purposes.

The way I imagine it, where currently the window of "system prompt" is inside the chat, and under the text input field, there would be a "save" button. When pressed, a title will be requested.
The saved preconfigured system prompts will be seen as a list of "cards/buttons" in the same system prompt window, under their assigned titles, below the "save" button, and above "restore default/ok" buttons. The preconfigured titled system prompts would be clickable. It could work by automatically filling the text field in the window with the saved text of the clicked prompt, then the user presses "ok" like normally and it's ready to use.

[jegly]

Good idea, whats a example of a workflow you would use it for ? i have not used the prompt myself only to test translator features

[Jakarrrg]

You could save different prompts for different use cases, like a prompt "you are a professional translator/an academic helper for philosophy/an interlocutor with a socratic dialectic approach... ", or "if you have not enough information and you are not sure about something, do not make assumptions, just make follow up questions to the user" etc. The potential uses are infinite.

The problem is that I tried using it as it is now and I am not sure it actually "listens/follows" the prompts. I really cannot determine.

[jegly]

You could save different prompts for different use cases, like a prompt "you are a professional translator/an academic helper for philosophy/an interlocutor with a socratic dialectic approach... ", or "if you have not enough information and you are not sure about something, do not make assumptions, just make follow up questions to the user" etc. The potential uses are infinite.

The problem is that I tried using it as it is now and I am not sure it actually "listens/follows" the prompts. I really cannot determine.

i will get onto it literally spent over 5 hours trying to get LaTeX rendering working correctly

[jegly]

You could save different prompts for different use cases, like a prompt "you are a professional translator/an academic helper for philosophy/an interlocutor with a socratic dialectic approach... ", or "if you have not enough information and you are not sure about something, do not make assumptions, just make follow up questions to the user" etc. The potential uses are infinite.

The problem is that I tried using it as it is now and I am not sure it actually "listens/follows" the prompts. I really cannot determine.

QuantSageX opened 2 days ago · edited by QuantSageX

Dynamic Max Context Extension Using Graph RAG but all chats should be stored on device... So even if you are chatting with it ...You should in theory have infinite context length if possible it should also incorporate https://github.com/MemMachine/MemMachine ... To make the burden lighter

I have no idea how easy it is to implement, but I have seen another open-source LLM app using it (https://github.com/timmyy123/LLM-Hub) and I think that, except from the other various advantages it has by definition, it would be a very useful tool to help interconnect the different functions/tools and improve the workflow by far, making the whole app experience feel more unified. Examples: 1) On audio scribe/voice input tools, after the result/text was given, there could be a simple button "load to global memory" (a title could be assigned as well to keep the different documents distinguished) or something like that, that would do just that. The user then could go to the chat tool and just ask for example "give me a summary of the text under the title AUDIO SCRIBE". 2) On agent skills, the possible use cases are as many as the skills are. The outcomes of most of the skills would be useful to be uploaded to global memory in order to analyse them in a conversation with the LLM model in chat tool. Translations, budget tracker data. All those could be uploaded to global memory and be readily accessible in the chat tool. I have even found a skill that can fetch text from sites (it is still a bit rough) so there the possibilities are many. As the models and skills continue to evolve it would make more sense to have this kind of functionality.

It could be something to keep in mind, if it sounds difficult right now.

i will aim to get this in 1.0.8. im sure bugs will be found in 1.0.7 :|

[RubenKelevra]

I was wondering if there's a chance to add an Ollama API server to the app.

This would allow us to use the phone as LLM "compute node" for a Laptop over WiFi.

Safety procaution which I think make sense:

• Feature is turned off by default, you need to go into the settings to turn it on.
• The API server is not started, when the app is started, but it also need an explicit tap on "start API server" in the context menu.
• When tapping start it will default to loading one model you need to select in the setting. And if you want to load anything else, it will be required to turn the turn the API server off again.
• The API server stays active unless the app is no longer in the foreground or the network is changed, in both cases the API server turns itself off.

[jegly]

I was wondering if there's a chance to add an Ollama API server to the app.

This would allow us to use the phone as LLM "compute node" for a Laptop over WiFi.

Safety procaution which I think make sense:

* Feature is turned off by default, you need to go into the settings to turn it on.

* The API server is not started, when the app is started, but it also need an explicit tap on "start API server" in the context menu.

* When tapping start it will default to loading one model you need to select in the setting. And if you want to load anything else, it will be required to turn the turn the API server off again.

* The API server stays active unless the app is no longer in the foreground or the network is changed, in both cases the API server turns itself off.

a lot of peeps seem to want this feature, its a big feature ! - i will look into it. it needs hard off switch. i dont want a http server running in the background lol

[jegly]

Global Memory / RAG — wont be in 1.0.8. True RAG needs an embedding model running on-device, a vector store, and a retrieval
pipeline. That's a separate ML model download, significant CPU/RAM cost, and a lot of new infrastructure. The MemMachine/Graph

RAG angle @QuantSageX suggested, A simple "save to clipboard, paste into chat" feature is feasible for
1.0.9, but calling it RAG would oversell it

Ollama API server — definitely not 1.0.8. Running an HTTP server in an Android app serving the Ollama REST API format is a
substantial project. LiteRT-LM has no streaming server interface so you'd be wrapping it yourself. Your instinct about the hard
off switch is right — a backgrounded HTTP server is a real concern. This is a 1.1.x feature at earliest.

• Saved system prompts will be in 1.0.8

[jegly]

#RoadMap#

1. Core AI Capabilities & Models

   Better multimodal support — Expand beyond current vision + text. Add audio understanding (e.g., analyze uploaded voice notes or music for genre/mood/lyrics transcription) and video understanding (short clips → summarize or answer questions about them).
   Advanced local agents / tool use — Improve function calling with more built-in skills:
   Local calendar/tasks integration (read-only for privacy).
   File system tools (search, summarize documents/PDFs in a folder).
   Device sensors (use accelerometer, GPS, battery, etc., with explicit permission).
   RAG (Retrieval-Augmented Generation) — Local vector database (e.g., via Lantern or simple embeddings + SQLite) so users can feed personal notes, books, or knowledge bases and query them.
   Model management upgrades:
   Quantization helper (on-device conversion options).
   LoRA fine-tuning support (even simple/low-rank for personalization).
   Automatic model benchmarking + recommendations based on device specs.
   Easier GGUF merging or merging tools integration.

2. Voice & Interaction

   Better TTS — More voice options (Piper + Coqui or local XTTS-style ).
   Interruptible real-time voice — True streaming where the AI can be interrupted mid-sentence naturally.
   Multilingual voice improvements — Better code-switching and accent handling in Whisper + TTS.
   Voice commands for app control — “Open camera”, “Switch to Gemma”, “Generate image of…”, etc.

3. Image & Media Generation

   Image editing / inpainting — Upload an image + prompt to modify it (very useful with stable-diffusion.cpp).
   ControlNet / IP-Adapter style guidance — Pose, depth, or reference image support.
   Batch generation + upscaling — Generate variations + simple 2x/4x upscaler.
   Video generation (ambitious) — Even short clips via models like AnimateDiff or stable-video-diffusion (if they fit on high-end phones).

4. Privacy & Security

   End-to-end encrypted backups — Export/import chat history securely (e.g., to another device via QR or password).
   App sandboxing / work profile mode — Optional isolated profile.
   Audit log enhancements — More detailed “what model saw/processed” logs.
   Permission hardening — Granular toggles and one-time permissions for camera/mic/file access per conversation.

5. Usability & Polish

   Multi-model chatting — Side-by-side comparison or routing (e.g., fast model for simple queries, smart model for complex ones).
   Custom system prompts / personas — Easy switching between different “characters” or instruction sets.
   Widget & quick tiles — Quick chat widget, “Ask Box” quick settings tile.
   Offline-first documentation — Built-in searchable help + model cards with performance estimates.
   Theme & accessibility — Full Material You dynamic colors, high-contrast mode, TalkBack optimizations.
   Export options — Markdown/PDF export of conversations, image galleries.

6. Performance & Hardware

   Adaptive inference — Dynamically switch accelerators or quant levels based on thermal state/battery.
   Sustained performance improvements — More aggressive thermal/clocks management.
   Memory optimization — Layer sharing or speculative decoding improvements across engines.
   Wear OS / foldable support — Companion watch app for voice, or optimized UI for foldables/tablets.

7. Ecosystem & Extensibility

   Plugin system — Community skills/plugins that can be downloaded (carefully sandboxed).
   Better integration with other local tools — e.g., sync with Obsidian vaults, Tasker, or Termux.
   Web UI / desktop companion — Optional local web server so you can use Box from a browser on the same network (for bigger screens).
   Federated model hub — Curated community GGUF/LiteRT models with verified hashes (inside the app).