You can do that. Most server vendors offer ESXi preinstalled on internal USB or SD cards.
Once booted, you don't have much action on the disk. When you use USB drive, ESXi doesn't create a permanent scratch partition, because the many write-operations of the log files would hurt the drive. So it's only stored in a RAMdisk and will be lost during a reboot.
You can manually create a persistant scratch location though. This KB tells you how: http://kb.vmware.com/kb/1033696
Regards
Tim