3D Scanning a Virtual Space

Assignment

• 3D scan an object (and optionally print it)

My Past Experience

I've been exploring with 3D scanning since 2018. That was before cloud 3D scanning services like Luma and Polycam were a thing. I became really good at Agisoft PhotoScan (later renamed to Agisoft Metashape) to scan objects. Here's a 3D model of a Chinese Penjing I scanned in Early 2019. The artist He Jianglong is an internet friend of mine from a PSP PSN game sharing community back in 2010.

The Idea

One of the most creative use of 3D scanning I've seen is Create Counter-Strike in UE4 by Quixel (a compnay focusing on 3D scanned assets, now owned by Epic Games). The video shows how they processed a walkthrough screen recording video of the iconic Counter-Strike De_Dust 2 map, and used photogrammetry to create a rough starting point for a detailed 3D model.

The result is a very detailed 3D model of the Counter-Strike De_Dust 2 map.

Counter-Strike De_Dust 2 map scanned by Quixel© Quixel

I really want to try this out with a virtual space I'm familiar with.

The Plan

I've spent a lot of time in Overwatch and I would test this technique with a map from that game.

I found a video on YouTube that anaylyze strategic positioning of the Blizzard World map for tank players.

Download the video

I downlaoded the video with yt-dlp, the best video downloader out there. I retrieved a mp4 file from the video's url. The file size is 1.68GB with a running time of 42 minutes 36 seconds.

The downloaded mp4 file

Import the video into Metashape

Agisoft Metashape is a photogrammetry software that I've used for a long time. I paid for the Russian company's lifetime license a long time ago, before the Ukraine war broke out. The software integrated a helpful workflow for importing videos. If you import a video, it will automatically generate a set of frames for you at the sampling configuration of your choice.

The sampling frames generated by Metashape

Unfortunately, the built-in video sampler is not very efficient. It estimated 6 hours to extract all the frames at "large" gap.

Metashape estimated 6 hours to extract frames

I decided to extract the frames with Premiere Pro. I sped the video up to 1000% (10x) and only extracted the first 50 seconds of the sped up video at 30 fps, which gives me around 1500 frames. This way I only need to process 3 frames, and it only took me about 10 minutes to process the frames. I also adjusted the resolution from 1920x1080 to 1920x900 to remove the game UI on the top of the screen.

Sped up the video in Premiere Pro

Back to Metashape, I imported the frames and proceeded to align all the photos. I deliberately did not use the "Sequential" option, since the sped up video is pretty jumpy and doesn't guarantee a good alignment in consecutive frames. The alignment rate is quite nice, 1406/1593 aligned (~88.2%). It took me aroun 5 minutes to align the frames.

Aligned the frames in Metashape, 1406/1593 aligned (~88.2%)

3D Scanning

Then I started to build the point cloud (dense cloud). At medium setting and medium size, it took around 10 minutes to build the point cloud. The point cloud looked promissing since you can tell some buildings and the floor level are. But there are many missing part in the high up area and the floor, since in the video, the player isn't staring at the ground or sky.

I then built the mesh from the point cloud. The mesh is not very good. I think building from this video is the limiting factor (not good enough coverage from a 3D scanning standpoint).

The model is exported as a glb file (90MB) and I uploaded it to Sketchfab so that you can see it below. I won't burden the CBA GitLab with this large file given how shitty the model looks.