The realm of AI has witnessed significant advancements what seems like almost overnight. One of the most remarkable being the development of conversational AI models like OpenAI’s GPT offering. I have been surprised by how rapidly things have advanced and feel as if I’ve been playing catchup. Playing catchup often looks like drinking from a firehose.
I strongly believe that we, as software engineers, should develop and understanding of and attempt to master these technologies. It can provide a substantial boost to your skillset, opening new paths for you to explore and innovate.
In this post, I aim to help you ramp up on ChatGPT and provide a roadmap to effectively get started in your own work.
First, I will point you to a blog series currently being published by Joseph Ours who leads the Modern Software Development Service Offering where I work at Centric Consulting. Joe is also heading up our AI/ML offering and you will definitely benefit from hearing from him.
1. Build a Foundation of Knowledge
A great place to start is to try to understand the technologies involved and how they work. You start by spending some time with ChatGPT.
ChatGPT is built on a Transformer-based language model, called the GPT (Generative Pre-trained Transformer) series. Also, understanding the basics of Natural Language Processing (NLP), the Transformer model, and the idea of transfer learning will provide a solid foundation. Some great resources to get started are:
- “What is ChatGPT Doing … and Why Does It Work” by Steven Wolfram. (read)
- “The Hundred-Page Machine Learning Book” by Andrly Burkov. (read)
- “Natural Language Processing in TensorFlow” course on Coursera. (found on internet, not taken)
- “The Illustrated Transformer” by Jay Alammar. (found on internet)
2. Get Hands-on
One you understand the basics find ways to start working in the tech. Get a license to ChatGPT from OpenAI and try some things out. OpenAI provides an easy-to-use API for its GPT-3 model. This is a great way to familiarize yourself with how to use these models practically. Visit OpenAI’s documentation to learn more about how to make API calls, set parameters, and interpret the results.
Microsoft’s Azure OpenAI Service is available in beta and is a great way to get some experience working in a sandbox and start to build tools.
3. Understand Use-cases and Applications
With a basis of understanding, and a knowlege of how to use the tools you may turn your attention to understanding how they can be used effectively. I like to look at possible problems to solve for people and businesses where this tech is particularly useful. How might people use ChatGPT to address problems they have or do not know they have.
There are obvious use cases from drafting emails to coding help, content generation to tutoring. There is also a broad range of potential business applications. Finding and reading case studies and example applications can provide a wealth of inspiration.
4. Learn to Fine-tune Models
While pretrained models like GPT-3 are powerful, they often need to be fine-tuned on specific tasks for optimal performance. Understanding how to fine-tune these models requires learning about concepts like transfer learning, data preparation for fine-tuning, and managing fine-tuning sessions.
5. Grasp Ethical Considerations and Usage Policies
It is important to understand and address the potential ethical considerations and implications, such as the risk of generating harmful or misleading content, privacy concerns, and more. OpenAI’s usage policies provide some guidelines here.
6. Keep Going
The field is rapidly advancing, and staying updated is crucial. Follow OpenAI’s blog and other AI news outlets, join AI communities on Reddit, Stack Exchange, and other platforms, and participate in conferences and workshops to keep abreast of new developments. Keep an eye on Centric’s ongoing blog series by Joseph Ours.
7. Experiment, Build, and Innovate
Finally, the best way to truly understand and master any technology is to use it. Experiment with different use-cases, build projects, create prototypes, and don’t be afraid to innovate. This is how we get to solutions that haven’t been considered yet. This is a new and exciting field with lots of room for creative and innovative. The more people spending time on this the better.