QDB namespace conversion process

Hi Crystal,

Indeed, the transition from bulkload to raft mode needs more polishing. :slight_smile: These are the steps for now, I’ll add a command to automate some of this:

  • After bulkload is done, shut down the instance and create a brand new QDB folder using quarkdb-create in a different location, using the nodes you’d like to have in the cluster.
  • Steal its raft-journal, and move it to the QDB directory of the bulkloaded instance. You can just delete the newly created QDB created directory then.
  • Manually copy the QDB directory with the data to all nodes you want to have in the cluster.
  • Make sure the configuration is now in mode “raft”, not “bulkload” for all nodes, and start them all up.

I’m more worried that it took four days… We calculated that even for our largest instances, it shouldn’t take more than a couple of hours. A few questions:

  • How many files and directories?
  • What write rates was QDB reporting during bulkload?
  • Which QDB version?
  • How capable is the machine on which you’re running QDB? (cores, RAM) Are you running the tool and QDB on the same machine?
  • How much time is actually spent writing into QDB, and how much time does it spend compacting?
  • And most importantly: Are you using an SSD? From our tests running the new namespace in a pre-production instance, we found out that an SSD is pretty much mandatory, otherwise performance will become very poor as soon as the total namespace size exceeds the physical RAM available on the machine.