Actually there is a simpler way - the Matrix4 class has a method which takes the camera pos, up and at vectors directly. And all as Vector4's.