Recently, I contributed to the open source project Chatcraft, and participated in the development of 10 release versions from v1.1.0 to v2.0.0. I would like to write a blog post to summarize my contributions.
New Feature contributions
1. Enables users to upload images to ChatCraft and work with vision-related models (related release: v1.2.0, v1.3.0, v1.4.0, v1.6.0, v1.9.0, v2.0.0)
Release note:
v1.2.0: first release
v1.3.0: supports mobile
v1.4.0: supports image compression
v1.6.0: supports multiple image compression options
v1.9.0: remove workaround hardcode
v2.0.0: supports GPT-4 Turbo
Demonstration
Note, although the message time displayed is current, here is the v1.2.0 commit branch being used to show the initial version functionality.
The latest release v2.0.0 is as follows, since the new model of gpt-4-turbo as of 04-09 also supports “Vision”, I have adapted it accordingly.
Users can also add images by using drag and drop and copy-paste methods.
drag and drop
copy-paste
To implement this feature, I studied the newly emerged vision-related API from the openai-node project and integrated it into the codebase, added new UI components to display user images, and made modifications to the model, database, and optimization operations such as image compression. For example, the integration of new API can be found in the changes from original type CreateChatCompletionRequestMessage to new type ChatCompletionMessageParam as below:
date: this.date.toISOString(),
type: this.type,
text: this.text,
+ imageUrls: this.imageUrls,
};
}
– toOpenAiMessage(): OpenAI.Chat.Completions.CreateChatCompletionRequestMessage {
+ toOpenAiMessage(model: ChatCraftModel): OpenAI.Chat.Completions.ChatCompletionMessageParam {
const text = this.text;
+
+ const content: OpenAI.Chat.Completions.ChatCompletionContentPart[] = [{ type: “text”, text }];
+ if (model.supportsImages && this.imageUrls.length > 0) {
+ this.imageUrls.forEach((imageUrl) => {
+ content.push({
+ type: “image_url”,
+ image_url: { url: imageUrl },
+ });
+ });
+ }
+
switch (this.type) {
case “ai”:
return { role: “assistant”, content: text };
case “human”:
– return { role: “user”, content: text };
+ return { role: “user”, content };
case “system”:
return { role: “system”, content: text };
case “function”:
@@ -119,6 +137,7 @@ export class ChatCraftMessage {
chatId,
type: this.type,
text: this.text,
+ imageUrls: this.imageUrls,
};
}
This new type suggests a more flexible structure that can accommodate multiple types of content within a single message. This is crucial for integrating capabilities for the image processing and handling, which are essential for models that support vision features.
The image compression function is completed client-side, which can reduce the cost of the user’s API token. Users can make related settings in the settings modal.
As a side effect, I use it to do quick image compress.
This feature has been the biggest challenge I’ve faced in the open-source community so far. In my previous post, ‘A 55-commit PR,’ I also wrote about my feelings after releasing. Thanks again to the ChatCraft community for their help and support.
2. Enables users to generate images with the DALL·E 3 model (related release:v1.5.0, v1.7.0, v1.8.0, v2.0.0)
Since the image generation model kind of independent than the language models, my approach is to use ChatCraft’s command interaction, and I added an /image command, usage:
layout=[l|landscape|p|portrait]
Demonstration
3. Supports rendering of mathematical expressions (LaTeX) (v1.1.0)
Demonstration
(a night mode example)
Improved UI contribution
Allows for the hiding and showing of long chat. (v1.1.0, v1.2.0)
It works great when the message is super long. During the development, I encountered a Minified React error issue. If you are interested, you can check my post ‘Minified React error #426‘ to see a possible solution. An interesting update in v1.2.0 is we don’t fold super long final message to give the user better experience.
Bug fixes contributions
1.dollar symbol render conflict vs Latex rendering issue (v1.1.0)
Bug
Fix
2.no label code block rendering issue (v1.1.0)
Bug
Fix
3.Html no syntax highlight issue (v1.2.0)
Bug
Fix
4.encode & in Markdown URLs issue (v1.2.0)
Bug
Fix
command failed, converting that to &
5.non-OpenAI model uses OpenAI logo (v1.8.0)
Bug
Fix
6. Disable message editing on shared chats (v1.9.0)
Bug
Fix
Reflections
My contributions to ChatCraft from versions v1.1.0 to v2.0.0 have been both challenging and rewarding. Throughout this journey, I have been deeply involved in enhancing the functionality and user experience of ChatCraft. These contributions have not only improved the platform but have also provided me with invaluable learning experiences.
My work on ChatCraft has been a profound learning curve and a testament to the power of community and collaboration in the open-source space. I am grateful for the support from the ChatCraft community, which has been instrumental in overcoming challenges and achieving milestones. I look forward to engaging more with the community and continuing to contribute to the evolution of ChatCraft.
