I'm doing the oppposite. I have some 3090's coming in a few days to build a dedicated AI server. I'm thinking about grabbing some A6000 though, as they perform slightly better than 2 3090's and have similar ram.

It really depends on what you want to do, if you want to be able to use a lot of models, and different tools, having your own hardware is key, especially during development. Deployment, you are almost always going to need to use cloud.

Hardware can give you access to near ChatGPT 4 quality models for relatively cheap. The next step is up is astronomically more expensive though. Dual nVidia 3090 (about $800-900 each on eBay) will allow you to use 70B parameter models, this is near top tier for open source models.

Macs allow you to get to the next step a lot cheaper, but with limits. The Mac Studio for example will allow for 192GB of unified ram which will allow you to run models with ~148G size for about $7000.

You probalby won't save on API fees (unless you are doing a ton) but you will have a lot more options.