Today, Microsoft announced that you'll now be able to close your Xbox Insider account if you choose to do so, right through the Xbox Insider Hub app. You might wonder why one would do so, when you can simply remove your console from the program.
The answer would be to remove all of your Xbox Insider data. Once you leave the program, it will all be deleted, but that also means that your points will be gone. If you rejoin the Xbox Insider Program, you'll have 0 XP and you'll be at Level 1.
So upon rejoining, you'll only be able to enroll your console in the Preview Omega ring, which is the Xbox equivalent to the Windows Insider Program's Release Preview ring. You'll have to work your way up through the program by completing Quests and surveys, and you'll be able to join Delta and Beta after certain periods of time. Alpha is still by invitation-only.
You can find the option to close your account through Settings -> Manage Account. If you want to download all of your data before closing the account, you can do so here.