On 12/1/2017 11:32 AM, hw wrote: > So this would mean that the database (running on a different server) > takes > almost two times as much as foo --- which I would consider kinda > excruciatingly > long because it´s merely inserting rows into two different tables > after they were > prepared by foo and then processes some queries to convert the data. > > The queries after importing may take like 3 or 5 minutes. About 4.5 > million rows > are being imported. so you're missing about 25 minutes, and maybe 5 minutes is spent post processing, so thats 20 minutes spent in the data insertion? inserting one row at a time? or in batches? remeber a database server is going to do commits after each transaction, which forces the data to be flushed to disk. 4.5 million seperate row transactions, yeah, I could see that taking some time, plus add that many network round trips, etcetc. if the db server just has a single SATA disk, you're doing 9 million committed writes combined to the two tables? 20 minutes for 9 million inserts, thats 7500 per second. -- john r pierce, recycling bits in santa cruz