When you use Siri, your commands are uploaded to Apple’s servers. Apple assigns a random number to the command, which is associated with the voice file. This random number gets stored on the backend, not your Apple ID or e-mail.
If you turn Siri off, Apple will remove the number from your Siri files. This completely disconnects their stored data from you. The sound files will remain on Apple’s servers for about 18 months for testing and product improvement before it gets deleted. The voice files are simply used just to improve Siri.