Vast we once were

X never marked a particular spot for us

Merciless we claimed much!

The Romans.

This answer refers to the ancient Romans that mercilessly conquered its neighbors and grew into a vast empire...and, it also refers to the Roman numerals.

"I" is singular, representing "one".

The other numerals stand for larger quantities, so plural pronouns are appropriate.