Search for a command to run...
"Bumpy roads lead to beautiful places"
No comments yet. Be the first to comment.
This series will walk through the steps in order to install a local development environment that includes Oracle 23ai, ORDS & APEX.
If you’re working with object storage on OCI at all, you’ll notice that the Oracle 23ai Free container database is missing a crucial component: DBMS_CLOUD. DBMS_CLOUD is a package provides a layer so that the database can manage files and directories...
I saw Dimitri's post on SQL Assertions today and it piqued my curiosity. Putting more business rules closer to data is almost always a great idea, as you can now call DML from any technology and rest
Almost 20 years ago to the date, I submitted my resignation to Oracle to go start my own company. It was a difficult decision to make, as APEX - or HTML DB as it was called then - was just starting to gain traction. However, I was presented with an o...
One of the more difficult parts of the software development lifecycle is deployment. While it seems simple on the surface - just write some scripts and run them - it gets pretty complicated pretty quickly as the level of sophistication of your system...
Another AI-centric tool that has raised a lot of interest is Claude Desktop. Like Cline, Claude Desktop uses the same reasoning engine at the core - Claude Code - to assist with development tasks. While they share the same engine, there are use cases...
When I first heard the term MCP used in the context of AI, I was immediately taken back to the 80s when the movie Tron was released. For those who have not seen it, Tron is a futuristic movie about a curious user who is literally sucked into a comput...
Unless you’ve been living under a rock, you’ve likely heard about AI & LLMs more than once.
LLM stands for Large Language Model. It's a type of artificial intelligence (AI) model that's designed to process and understand human language at a massive scale. Large Language Models are trained on vast amounts of text data, which enables them to learn patterns, relationships, and structures within language. This training allows the models to generate human-like text, respond to questions, and even engage in conversations.
And to be fully transparent, an LLM (Llama 3.2) wrote that last paragraph. In fact, AI generates the SEO summaries for all of my blog posts, as it’s a feature of the Hashnode blogging platform. Ya, it’s everywhere.
So why not integrate it with our local development platform?
In this post, we’ll walk through the steps to download & install an LLM - Llama 3.2 - to our local machine, fire it up and integrate it with APEX in our Oracle 23ai container.
The first thing that we need to do is download a program called Ollama. Simply put, Ollama is an application that enables users to install and run LLMs on their local machines. It obfuscates all of the hard parts, and in fact, the next few steps will only take a few minutes to complete.
To download Ollama, follow the steps below:
Navigate https://ollama.com/
Click Download.
Select your operating system and the download will start.
Once it’s downloaded, unzip the file and drag the Ollama icon to your main Applications folder.
Double-click the Ollama icon to start the installation. You should see the following window:
Click Next.
Ollama will prompt to install the command portion. You will need to enter an Administrator password to pass this step after you click Install.
ollama run llama3.2 from the window by clicking on the copy icon.
Click Finish to dismiss the window.
Open a new terminal window.
Paste in the command - ollama run llama3.2 - to install the llama3.2 model and hit return. The model will begin to download. Depending on your network speed, this may take a few minutes, as the size of the download is roughly 2GB.
>>> prompt, enter any question that you want to ask the model. For example: what does llm stand for?
Ollama is now up and running with the Llama 3.2 LLM. This is the same exact model that you may have interacted with online, but it’s running locally on your machine and is completely free!
Feel free to play around. When you want to exit the interactive mode, type /bye and hit return.
Since we “ran” a model, Ollama will check to see if it has the latest model, pull it down if not, and then run the model. You can check to see if it’s running with the following command: ollama ps
If we wanted to stop this model, we can use the following command:ollama stop llama3.2
This will stop the model, but Ollama will technically still be running. To completely quit Ollama, select the Ollama icon in the menubar and select Quit Ollama.
This will stop all models, as well as any of the Ollama background processes.
While this is all well and good, it’s really not much different from interacting with a LLM via a web site. In fact, if we wanted to go that direction, we could install Open WebUI and get a very familiar looking interface.
But that’s not what we’re after. We want to be able to use this LLM in APEX to drive the APEX Assistant as well as the APEX_AI API. To do that, it may make more sense to run the LLM as a service.
ollama serve
curl --request POST \
--url http://localhost:11434/api/chat \
--header 'content-type: application/json' \
--data '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "who was the first president?" }
],
"stream": false
}'
The results should look similar to the following:
{
"model": "llama3.2",
"created_at": "2024-12-05T04:03:00.706654Z",
"message": {
"role": "assistant",
"content": "The first President of the United States was
George Washington. He served two terms in office
from April 30, 1789, to March 4, 1797."
},
"done_reason": "stop",
"done": true,
"total_duration": 1633274834,
"load_duration": 18144875,
"prompt_eval_count": 31,
"prompt_eval_duration": 667000000,
"eval_count": 36,
"eval_duration": 946000000
}
Note that the content line contains the answer from the LLM. Thus, the Ollama service is up and running.
Given that we want to reach out to this service from a Podman container, we can’t use localhost, as that would translate to the container itself. Thus, we need to refer to the service on the IP address of the local machine - i.e. your local IP address.
To find out your local IP on Mac:
Open System Settings > WiFi > Details.
From there, either scroll down or select TCP/IP to see the IP address of your local machine. Note this address, as we will need it in the next step.
[your IP address] with you local machine’s IP address and run it:curl --request POST \
--url http://[your IP address]:11434/api/chat \
--header 'content-type: application/json' \
--data '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "who was the first president?" }
],
"stream": false
}'
You should get the same or very similar result as you did before.
If all is well, then we’re ready to hook up the LLM to APEX.
In some cases, you may need to run the following command on the Mac, and then stop & start Ollama:
launchctl setenv OLLAMA_HOST "0.0.0.0"
This essentially “unbinds” the Ollama service from 127.0.0.1, allowing you to use your local IP address to access it.
Integrating Ollama with APEX is done pretty much the same way as a hosted LLM, with a couple of small differences. Let’s walk through the steps.
First things first, we need to create an additional ACE entry to allow the database to talk to the Ollama service.
Open a new terminal window.
Connect to the database via SQLcl as the SYS user.
Ensure that you’re connected to the correct PDB. In this case, it should be freepdb1.
Copy the below command, replacing [your IP address] with you local machine’s IP address and run it:
begin
dbms_network_acl_admin.append_host_ace (
host => '[your IP address]',
lower_port => 11434,
upper_port => 11434,
ace => xs$ace_type(privilege_list => xs$name_list('connect'),
principal_name => 'APEX_240100',
principal_type => xs_acl.ptype_db));
end;
/
If the IP address of your local machine changes, you will need to create another ACE entry for each new IP address. Thus, it may be easier to use * as the hostname, as that will apply to any and all IP address and/or hostname.
Again, this idea is fine for a local development server, but not for production.
We can quickly test this via SQL Workshop:
Login to any APEX workspace (except INTERNAL).
Click on SQL Workshop.
Click on SQL Commands.
Copy the below command, replacing [your IP address] with you local machine’s IP address and run it:
declare
l_clob clob;
begin
l_clob := apex_web_service.make_rest_request
(
p_url => 'http://[your IP address]:11434/api/chat'
,p_http_method => 'POST'
,p_body => '{ "model": "llama3.2", "messages": [ { "role": "user", "content": "who was the first president" }], "stream": false}'
);
htp.prn(l_clob);
end;
/
If all is working, you should see something similar to the following:
Again, the results may differ slightly, but as long as there’s a JSON document that looks similar to the one above, you’re good to go.
If there’s an error here, it almost always has to do with the ACE being improperly configured or the IP address being wrong. Verify that both of these components are correct and try again.
Integrating our model with App Builder is done pretty much the same way as you would any other hosted LLM. The only real difference is that we’re using our local machine’s IP and the value of the API Key really doesn’t matter, but can’t be null.
Let’s get this thing integrated!
Let’s start by adding our LLM to the workspace.
Click the App Builder icon.
Click the Workspace Utilities icon.
Click Generative AI.
Click Create.
Set or enter the following values, replacing [your IP address] with you local machine’s IP address:
| Attribue | Value |
| AI Provider | Open AI |
| Name | Llama 3.2 |
| Static ID | llama_32 |
| Used by App Builder | [selected] |
| Base URL | http://[your IP address]:11434/v1 |
| Credential | - Create New - |
| API Key | xxxxxxxxx |
| AI Model | llama3.2:latest |
Note: The value of API Key can be anything, as it’s not needed when using local LLMs, but cannot be null.
Let’s ensure that it’s working.
Navigate to SQL Workshop > SQL Commands.
Click on the APEX Assistant button.
Since we may not have any tables in our schema, simply type hello and hit enter. You should see a similar response:
While this is less than helpful, it does prove that APEX is able to talk to Ollama.
Rather than re-invent the wheel, there’s a decent amount of content on the internet about how to build applications with APEX & AI - using chat, generation and the APEX_AI API. Now that we have our local LLM integrated, there’s nothing different that needs to happen when building APEX application that interact with AI - either via a dynamic action or API call.
Here’s a few links to get started:
AI offers an unparalleled set of features that can be used as a platform to build the next generation of application on. APEX provides an ideal place to start to quickly hash out, prototype and build these solutions. The ability to pack the power of AI onto your local machine can accelerate development and save cost at the same time.
Title Photo by Chris on Unsplash