You need to use virtualenvs.  Should be something like sudo apt install python3-virtualenv or something.  You can find lots of docs on this with Google.  It's probably breaking because both apt and pip want to manage Python packages, but installing srtuff into the global Python environment isn't a good idea at all.

Ideally you also use requirements.txt which, again, lots of docs via Google.

Virtual environments are what you should use but you can also use --user when installing stuff and that'll put packages in your user directory. Then apt or whatever package manager can manage global python packages and it won't wine about user-installed packages via --user. You can also go really far and use pyenv to manage packages and python installations (but that may not give you access to system-wide packages).

Thank you for all the advice!
Another question, though. Does anybody know of a way I can either install ubuntu server accessibly, or completely automate the thing? I'm finding out lots of stuff isn't working after lots of not being used, so figure a reinstall and setitng things up fresh will help.
If this isn't possible with ubuntu, I'm also fine with other suggestions for a good server distro. This isn't something I know much of anything about, as it's been a while since I've given the one I use a thought at all.

Orca is built into their graphical installer; you need speakers and either windows+s or alt+s to turn it on.  If the server distro you're using doesn't have it, just do normal desktop Ubuntu and don't use the desktop part.  Server distros aren't special, they're just stripped down by default, and as I recall the Ubuntu installer will let you tell it not to install a desktop anyway.

I'd link you the docs on this but I don't have the link and it's incredibly hard to find via Google, plus all they really say is "wait for the installer then hit alt and/or windows plus s".  It's one of those, I just don't remember if it's alt or windows right now.