Possible to do the whole calculation using 3d points, but it is more efficient to use 2d points.
Converting to the coordinates in the plane is computationally expensive. Idea: project to a coordinate plane (XY, YZ, or XZ) by discarding one of the vector coordinates.
We cannot always discard, say, Z, because the polygon may project to an interval, if it is in a plane parallel to Z.
Choose the coordinate to discard so that the corresponding component of the normal to the polygon is maximal. E.g. if nx > ny and nx > nz,discard X.