Split public LLM provider code from internal (#734)

Split public LLM provider code from internal (#734) #805