Close
0%
0%

Build Local LLM agent on Raspberry Pi 5

I have build a local LLM Agent on my Raspberry Pi 5 with M.2 NVME SSD & PoE + Hat

yoyojackyyoyojacky
Following
Liked Like project

Just one more thing

To make the experience fit your profile, pick a username and tell us what interests you.

Pick an awesome username
hackaday.io/
Your profile's URL: hackaday.io/username. Max 25 alphanumeric characters.
Pick a few interests
Projects that share your interests
People that share your interests

We found and based on your interests.

Choose more interests.

OK, I'm done! Skip
Similar projects worth following
678 views
0 comments
5 followers
View Gallery
As we know, we can attach a M.2 NVMe SSD via PCIe expansion board, and connect it via FFC cable to Raspberry Pi 5 on PCIe slot, it will provide high perfornance on data transfering and speed improvement. and at the same time, the local large language model is more and more popular right now, I want to build my own Local LLM server, so I've tried installed ollama on my Raspberry Pi 5 and pulling down llama3 and phi3 model to build my own AI agent, it works fine, although it is a little bit slow, but I have purchased Hailo-8L AI co-processor for further development. BTW, I almost forot the PoE hat, it will provides 5v@4.5A power to my Raspberry Pi via Ethernet cable which connected to my Meert AYDLINLATMA PoE Power Supply( OUTPUT： 48V/1A 48W 802.3at protocol)
It works fine and I am going to build webUI such as lobe-chat or other UI to deploy it on my local network.

KZ-0066-01.jpg

JPEG Image - 973.14 kB - 06/18/2024 at 08:09

Preview

KZ-0066-05.jpg

JPEG Image - 852.90 kB - 06/18/2024 at 08:09

Preview

KZ-0066-02.jpg

JPEG Image - 1002.68 kB - 06/18/2024 at 08:09

Preview

KZ-0066-08.jpg

JPEG Image - 852.15 kB - 06/18/2024 at 08:09

Preview

KZ-0066-09.jpg

JPEG Image - 904.83 kB - 06/18/2024 at 08:09

Preview

View all 10 files

Enjoy this project?

Share

Discussions

Similar Projects

newest edition of its Power over Ethernet (PoE) HAT. The Raspberry Pi PoE+ HAT outperforms its predecessor by adopting the 802.11at PoE+...
Project Owner Contributor

The Raspberry Pi PoE+ HAT Has Been Announced.

undefined

"Allows for the safe and orderly shutdown of a Raspberry Pi and any attached hardware"
Project Owner Contributor

UPS HAT For Raspberry Pi - SB Components

undefined

Can Raspberry PI with 8GB of ram can handle WINDOWS 10 OS. Let's get into a honest review with my system specs.
Project Owner Contributor

Windows 10 review on raspberry pi 4

lithium-ionLithium ION

Build HAT For Raspberry Pi Has Been Officially Announced
Project Owner Contributor

Build HAT for using Pi and LEGO® Education

undefined

Does this project spark your interest?

Become a member to follow this project and never miss any updates

Going up?

About Us Contact Hackaday.io Give Feedback Terms of Use Privacy Policy Hackaday API

© 2024 Hackaday

Yes, delete it Cancel

Report project as inappropriate

You are about to report the project "Build Local LLM agent on Raspberry Pi 5", please tell us the reason.

Send message

Your application has been submitted.

Remove Member

Are you sure you want to remove yourself as a member for this project?

Project owner will be notified upon removal.