[python] sqlite: Can you combine two database files?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
[python] sqlite: Can you combine two database files?
I have a database for all the files on a disk. I have several disks. Sometimes I want to rescan one disk but not the others, so I keep each disk's database in its own file, i.e. disk1.db, disk2.db, disk3.db, etc.
is there a quick way to combine all these databases so that I can make one SELECT call for all db's? I know I could pre-process all the db's into one memory db by doing a "select *" on each & then inserting the results into the memory db, but I'm hoping to find something a little quicker/easier/ more efficient/correct.
If it matters, I'm doing this in python2.5 with the sqlite3 module.
It's not a one-off. The system is a bit more complex than I alluded to (the "disks" are actually multi-terabyte arrays and the db files may be generated by deamons at regular intervals), but principle is the same - merge two or more dbs together & use the combined db for select statements.
At the end of the day, I may still do what you suggested, as it's the obvious solution (even if done on the fly), but I was hoping for something a little more clever or by design.
I'm not certain if sqlite supports the schema naming convention within the select (MySQL does) so you would do do something like SELECT * FROM disk1db.name, disk2db.name; However I'm guessing that you would then end up with a Cartesian join, can't you just tell I've never done this. Probably because I would have approached the problem differently by adding an extra field in the table that would identify the disk. That way a SELECT * FROM diskdb WHERE disk = disk1 would return the details from one disk and SELECT * FROM diskdb would return everything.
Keeping an extra column to identify disk/host would not solve my problem. What I need to do, ideally, is "attach" (combine) tables/databases/database_files from geographically/logically distributed hosts (so that I don't have to continually transfer large data from the remote hosts to the central host).
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.