Yes - scary is right.
As broken pointed out - CO is "stickier" than O2, starts to clog up the carrying sites on your hemoglobin until there aren't enough left to keep you alive.
Even after you stop exposure it takes time for the sticky CO to slowly dislodge and allow O2 to be carried again...