It might help if you consider spheres as a surface.
3D spheres are a 2D surface wrapped into a 3D space, likewise, hyperspheres would be a 3D surface wrapped into a 4D space. There's no "infinite spheres on its surface", I think the "rotation" is a better analogy.
Take a line rotated around an orthogonal axis and you have a circle, a circle rotated around an axis orthogonal to the other two is a sphere, a sphere rotated around another orthogonal axis is a hypersphere.
3D spheres are a 2D surface wrapped into a 3D space, likewise, hyperspheres would be a 3D surface wrapped into a 4D space. There's no "infinite spheres on its surface", I think the "rotation" is a better analogy.
Take a line rotated around an orthogonal axis and you have a circle, a circle rotated around an axis orthogonal to the other two is a sphere, a sphere rotated around another orthogonal axis is a hypersphere.