Display and operate 3D shape data in C # and save the image.

7 minute read

Overview

I couldn’t find a good template to easily display and operate 3D shapes and 3D models in C # and save them as images, so I will introduce my own method. In this article, we will use the OpenGL wrapper ** OpenTK ** for 3D display, the OpenCV C # wrapper ** OpenCVSharp ** for saving image data, and the ** SurfaceAnalyzer ** for loading 3D shapes. The completed image will be the following gif. First, read the 3D shape, rotate it with the mouse, and save it as an image.
The source code used this time is summarized in git.
AGD_20200814_205618_1.gif

Preparation

First, install the required libraries.

Creating a Windows Form Application

This time, we will create it as a Windows Form Application.
image.png

Installation of OpenTK, OpenTK.GLControl

OpenTK is a library for handling OpenGL in C #. GLControl is a control for displaying the OpenGL screen on the form.
image.png

Installation of OpenCVSharp4, OpenCVSharp4.runtime.win

OpenCVSharp4 is a wrapper for using OpenCV in C #. This time, it is used to save the image, so if you do not need to save the image, you do not need to install it. System.Drawing.Bitmap is fine, but we will use this one, which is versatile in consideration of applying post-processing of the acquired image. In order to display the image in a window, it is necessary to install the runtime suitable for each environment. For Windows, install OpenCVSharp.runtime.win.
image.png

Install Surface Analyzer

Surface Analyzer is a library for handling STL data in C #. For a detailed explanation, see the article here. Other libraries are fine as long as you can read the STL file.
image.png

Now let’s start coding. In this article, we will code the “Viewer form” ** that actually displays the shape and the “Control form” ** that controls the display.

Viewer form coding

 

Creating a Viewer form

This time, we will create a dedicated form to display the 3D shape and operate it from the Control form. Click Program Name> Add> Windows Form.
image.png

Viewer settings

Add GLControl as shown below and add an event. [Here] I refer to the () site. ** GLControl can be added on the designer side as well, but it doesn’t work for some reason **, so I use this method. The added event is for operating the camera on the screen. Right-click and drag rotate, and control the distance from the center of rotation by rotating the wheel.

Viewer.cs_GL Control system


using System;
using System.Drawing;
using System.Windows.Forms;

// openTK
using OpenTK;
using OpenTK.Graphics;
using OpenTK.Graphics.OpenGL;

// surfaceAnalyzer
using SurfaceAnalyzer;

namespace _3dview
{
    public partial class Viewer : Form
    {
        #region Camera__Field
        
        bool isCameraRotating;      //Whether the camera is rotating
        Vector2 current, previous;  //Current point, previous point
        float zoom = 1.0f;                 //Magnification
        double rotateX = 1, rotateY = 0, rotateZ = 0;//Movement by rotating the camera
        float theta = 0;
        float phi = 0;

        #endregion

        public Viewer()
        {
            InitializeComponent();

            AddglControl();
        }

        //Add glControl
        GLControl glControl;
        private void AddglControl()
        {
            SuspendLayout();

            int width = this.Width;
            int height = this.Height;

            //Initialization of GLControl
            glControl = new GLControl();

            glControl.Name = "SHAPE";
            glControl.Size = new Size(width, height);
            glControl.Location = new System.Drawing.Point(0, 0);
            glControl.SendToBack();

            //Event handler
            glControl.Load += new EventHandler(glControl_Load);
            glControl.Resize += new EventHandler(glControl_Resize);
            glControl.MouseDown += new System.Windows.Forms.MouseEventHandler(this._3DView_MouseDown);
            glControl.MouseMove += new System.Windows.Forms.MouseEventHandler(this._3DView_MouseMove);
            glControl.MouseUp += new System.Windows.Forms.MouseEventHandler(this._3DView_MouseUp);
            glControl.MouseWheel += new System.Windows.Forms.MouseEventHandler(this._3DView_MouseWheel);

            Controls.Add(glControl);

            ResumeLayout(false);

        }

        private void glControl_Load(object sender, EventArgs e)
        {
            GLControl s = (GLControl)sender;
            s.MakeCurrent();

            GL.ClearColor(Color4.White);
            GL.Enable(EnableCap.DepthTest);

            Update();
        }

        private void glControl_Resize(object sender, EventArgs e)
        {
            GL.Viewport(0, 0, glControl.Size.Width, glControl.Size.Height);
            GL.MatrixMode(MatrixMode.Projection);
            Matrix4 projection = Matrix4.CreatePerspectiveFieldOfView((float)Math.PI / 4,
                (float)glControl.Size.Width / (float)glControl.Size.Height, 1.0f, 256.0f);
            GL.LoadMatrix(ref projection);

            Update();
        }

        private void _3DView_MouseDown(object sender, System.Windows.Forms.MouseEventArgs e)
        {
            //When the right button is pressed
            if (e.Button == MouseButtons.Right)
            {
                isCameraRotating = true;
                current = new Vector2(e.X, e.Y);
            }
            Update();
        }


        private void _3DView_MouseUp(object sender, System.Windows.Forms.MouseEventArgs e)
        {
            //When the right button is pressed
            if (e.Button == MouseButtons.Right)
            {
                isCameraRotating = false;
                previous = Vector2.Zero;
            }
            Update();
        }


        private void _3DView_MouseMove(object sender, System.Windows.Forms.MouseEventArgs e)
        {
            //When the camera is rotating
            if (isCameraRotating)
            {
                previous = current;
                current = new Vector2(e.X, e.Y);
                Vector2 delta = current - previous;
                delta /= (float)Math.Sqrt(this.Width * this.Width + this.Height * this.Height);
                float length = delta.Length;

                if (length > 0.0)
                {
                    theta += delta.X * 10;
                    phi += delta.Y * 10;
                    rotateX = Math.Cos(theta) * Math.Cos(phi);
                    rotateY = Math.Sin(phi);
                    rotateZ = Math.Sin(theta) * Math.Cos(phi);
                }

                Update();
            }
        }

        private void _3DView_MouseWheel(object sender, System.Windows.Forms.MouseEventArgs e)
        {
            float delta = e.Delta;

            zoom *= (float)Math.Pow(1.001, delta);

            //Enlargement / reduction restrictions
            if (zoom > 4.0f)
                zoom = 4.0f;
            if (zoom < 0.03f)
                zoom = 0.03f;

            Update();
        }
    }
}

Addition of 3D shape rendering method

Next, create a 3D shape display.
The ** Update () method ** is a method that is called every time the screen display is updated. From here we call the Render () method every time.
** Render () method ** is a method to change the screen display. The argument polygon will be the shape read by the Surface Analyzer. The screen display is operated by calling this method from the Control.cs side.
The ** DrawPolygons () method ** displays polygons of the read shape one by one. It can be displayed on the screen by giving colors, normals, and vertices between GL.Begin () and GL.End (). Here, the drawing color of the surface is specified according to the direction of the normal.
The ** N2TK () method ** transforms the System.Numerics and OpenTK Vector3 vectors.

Viewer.cs_Screen display



        PolygonModel Polygon;
        public void Update()
        {
            if (Polygon == null) return;
            Render(Polygon);
        }
        public void Render(PolygonModel polygon)
        {
            Polygon = polygon;

            //Clear buffer
            GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.DepthBufferBit);

            //Camera settings
            Vector3 vec_rotate = new Vector3((float)rotateX, (float)rotateY, (float)rotateZ);
            Vector3 center = new Vector3(N2TK(Polygon.GravityPoint()));
            Vector3 eye = center + vec_rotate * center.LengthFast / zoom;
            Matrix4 modelView = Matrix4.LookAt(eye, center, Vector3.UnitY);

            //Display settings
            GL.MatrixMode(MatrixMode.Modelview);
            GL.LoadMatrix(ref modelView);

            //3D shape display
            DrawPolygons(polygon);

            //Swap buffer
            glControl.SwapBuffers();
        }

        private void DrawPolygons(PolygonModel polygon)
        {
            if (polygon == null) return;

            //drawing
            GL.Begin(PrimitiveType.Triangles);

            //Draw a triangle
            for (int l = 0; l < polygon.Faces.Count; l++)
            {![Something went wrong]()

                var normal = polygon.Faces[l].Normal();
                GL.Color4(Math.Abs(normal.X), Math.Abs(normal.Y), Math.Abs(normal.Z), 0);
                GL.Normal3(N2TK(normal));
                GL.Vertex3(N2TK(polygon.Faces[l].Vertices[0].P));
                GL.Vertex3(N2TK(polygon.Faces[l].Vertices[2].P));
                GL.Vertex3(N2TK(polygon.Faces[l].Vertices[1].P));
            }
            GL.End();
        }

        // Numerics.Open TK Vector3.Convert to Vector3.
        private static OpenTK.Vector3 N2TK(System.Numerics.Vector3 vec3) => new Vector3(vec3.X, vec3.Z, vec3.Y);

Addition of imaging method

Reads the displayed contents of OpenTK pixel by pixel and converts it to Mat of OpenCVSharp. Since it is slow to read the data pixel by pixel, we use the method of copying the memory directly. In order to use Marshal, you need to add using.

Viewer.cs_Imaging


using System.Runtime.InteropServices;

        //Save image
        public OpenCvSharp.Mat GetMat()
        {
            int width = glControl.Width;
            int height = glControl.Height;
            
            float[] floatArr = new float[width * height * 3];
            OpenCvSharp.Mat ret = new OpenCvSharp.Mat(height, width, OpenCvSharp.MatType.CV_32FC3);

            //Loading images into dataBuffer
            IntPtr dataBuffer = Marshal.AllocHGlobal(width * height * 12);
            GL.ReadBuffer(ReadBufferMode.Front);
            GL.ReadPixels(0, 0, width, height, PixelFormat.Bgr, PixelType.Float, dataBuffer);

            //Load to img
            Marshal.Copy(dataBuffer, floatArr, 0, floatArr.Length);

            // opencvsharp.Conversion to Mat
            Marshal.Copy(floatArr, 0, ret.Data, floatArr.Length);

            //Discard
            Marshal.FreeHGlobal(dataBuffer);

            return ret;
        }

This completes the coding of the Viewer form.

Control form coding

Next, code the Control form.

Button placement

Add a button on the Control form and set it to “View Viewer”, “Show Shape”, and “Save” respectively.
image.png

View Viewer form

Double-click each button, add an event, and write as follows.
image.png

First, create the behavior when you click the “Show Viewer” button.

Control.cs on the way


using System;
using System.Windows.Forms;

namespace _3dview
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        Viewer viewer;
        private void button1_Click(object sender, EventArgs e)
        {
            viewer = new Viewer();
            viewer.Show();
        }
    }
}

3D shape display

Then read the shape and display it.

Control.cs


using System;
using System.Windows.Forms;

namespace _3dview
{
    public partial class Control : Form
    {
        public Control()
        {
            InitializeComponent();
        }

        Viewer viewer;
        private void button1_Click(object sender, EventArgs e)
        {
            viewer = new Viewer();
            viewer.Show();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            //Read shape
            var polygon = SurfaceAnalyzer.LoadData.LoadSTL(@"local\cube3_Pointed 2.STL", true);

            //Shape rendering
            viewer.Render(polygon);
        }

        private void button3_Click(object sender, EventArgs e)
        {

        }
    }
}

Run and click the Show Viewer button to display the viewer. Then click the “Show Shape” button to display the shape. You can rotate it by right-clicking and zoom in / out with the wheel.
Screenshot 2020-08-14 20.03.42.png

Imaging of 3D shapes

Finally, save the contents of the Viewer form.

Control.cs_Imaging and saving


using OpenCvSharp;
using System;
using System.Windows.Forms;

namespace _3dview
{
    public partial class Control : Form
    {
        public Control()
        {
            InitializeComponent();
        }

        Viewer viewer;
        private void button1_Click(object sender, EventArgs e)
        {
            viewer = new Viewer();
            viewer.Show();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            //Read shape
            var polygon = SurfaceAnalyzer.LoadData.LoadSTL(@"local\cube3_Pointed 2.STL", true);

            //Shape rendering
            viewer.Render(polygon);
        }

        private void button3_Click(object sender, EventArgs e)
        {
            //Get viewer image
            using (Mat mat = viewer.GetMat())
            {
                //Image display
                Cv2.ImShow("mat", mat);

                //Save image
                Cv2.ImWrite(@"local\mat.jpg ", mat * 256);
            }
        }
    }
}

Execute [F5] and click the buttons in the order of “View Viewer”, “Show Shape”, and “Save” to display the image and save it as it is. Since the reference of the coordinate axes is different between OpenGL and OpenCV, it will be upside down.
スクリーンショット 2020-08-14 20.33.12.png

Summary

In this article, I introduced how to read STL data in C #, display and operate it, and save it as an image. I hope it will be useful when you want to handle 3D shapes in C #.