I learned a similar lesson without the scrolls. Spent way too much time buffing alchemy and ended up with lots of game-breaking potions. My acrobatics buff potion was astronomically powerful and lasted practically forever, so my character launched into orbit and just never came down.