A Video Game Testing Method Utilizing Deep Learning

A Video Game Testing Method Utilizing Deep Learning

Mohammad Reza Taesiri, Moslem Habibi, MohammadAmin Fazli

Abstract

Computer video games must pass different types of tests before release. Yet most products in this multibillion-dollar industry still exhibit various compatibility problems when run on end consumers" computers. In this work, we propose a new automated testing method which utilizes deep convolutional neural networks to test video game compatibility with target runtime environments. This will result in better support for various computing environments that run video games and a reduction of the effort needed for testing them. Our method executes tests both on local computers and the cloud. Locally, a game tester will test the video game with normal testing routines. After that, these tests are automatically replicated on the cloud, running the video game on different environments. With the help of two convolutional neural networks, corrupted frames of the game containing artifacts are automatically discerned, and by comparing the local execution to the ones on the cloud, the corresponding problematic Draw Calls are determined. These are then used as a basis for comparison in order to determine the root cause of the graphical issue.

Keywords

Video Game Testing, Automated Testing, Software Testing, Deep Learning, Convolutional Neural Networks

References

  • [1] Sales of batman: Arkham knight’s pc version suspended on steam (update), http://www.polygon.com/2015/6/24/8842447/, [accessed 20December-2017] (2015).
  • [2] Y. J. Choi, Providing novel and useful data for game development using usability expert evaluation and testing, in: Computer Graphics, Imaging and Visualization, 2009. CGIV’09. Sixth International Conference on, IEEE, 2009, pp. 129–132.
  • [3] P. Moreno-Ger, J. Torrente, Y. G. Hsieh, W. T. Lester, Usability testing for serious games: Making informed design decisions with user data, Advances in Human-Computer Interaction 2012 (2012) 4.
  • [4] N. M. Diah, M. Ismail, S. Ahmad, M. K. M. Dahari, Usability testing for educational computer game using observation method, in: Information Retrieval & Knowledge Management,(CAMP), 2010 International Conference on, IEEE, 2010, pp. 157–161.
  • [5] C. Buhl, F. Gareeboo, Automated testing: a key factor for success in video game development. case study and lessons learned, in: Proceedings of Pacific NW Software Quality Conferences, 2012, pp. 1–15.
  • [6] K. Peterson, S. Behunin, F. Graham, Automated testing on multiple video game platforms, uS Patent App. 13/020,959 (Feb. 4 2011).
  • [7] C. Schaefer, H. Do, B. M. Slator, Crushinator: A framework towards game-independent testing, in: Automated Software Engineering (ASE), 2013 IEEE/ACM 28th International Conference on, IEEE, 2013, pp. 726–729.
  • [8] A. M. Smith, M. J. Nelson, M. Mateas, Computational support for play testing game sketches., in: Artificial Intelligence for Interactive Digital Entertainment (AIIDE), 2009 the Fifth International Conference on, AAAI, 2009, pp. 167–172.
  • [9] I. Zarembo, Analysis of Artificial Intelligence Applications for Automated Testing of Video Games, in: Environment Technologies Resources, 2019 Proceedings of the International Scientific and Practical Conference, Vol. 2, pp. 170-174.
  • [10] A. Nantes, R. Brown, F. Maire, A framework for the semi-automatic testing of video games., in: Artificial Intelligence for Interactive Digital Entertainment (AIIDE), 2008 The Fourth International Conference on, 2008.
  • [11] B. Chan, J. Denzinger, D. Gates, K. Loose, J. Buchanan, Evolutionary behavior testing of commercial computer games, in: Evolutionary Computation, 2004. CEC2004. Congress on, Vol. 1, IEEE, 2004, pp. 125–132.
  • [12] Y,Zheng, X. Xie, T. Su, L. Ma, J Hao, Z. Meng & C. Fan, Wuji: Automatic online combat game testing using evolutionary deep reinforcement learning. 2019, in: 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE , pp. 772-784.
  • [13] J. Bergdahl, C. Gordillo, K. Tollmar, & L. Gisslén, Augmenting automated game testing with deep reinforcement learning, 2020, in: 2020 IEEE Conference on Games (CoG) pp. 600-603.
  • [14] C. Ling, K. Tollmar & L. Gisslén, Using Deep Convolutional Neural Networks to Detect Rendered Glitches in Video Games, 2020, in: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 16, No. 1, pp. 66-73.
  • [15] A. Watson, Deep Learning Techniques for SuperResolution, in Video Games, 2020, arXiv preprint arXiv:2012.09810.
  • [16] C.-S. Cho, K.-M. Sohn, C.-J. Park, J.-H. Kang, Online game testing using scenario-based control of massive virtual users, in: Advanced Communication Technology (ICACT), 2010 The 12th International Conference on, Vol. 2, IEEE, 2010, pp. 1676–1680.
  • [17] Y. Choi, H. Kim, C. Park, S. Jin, A case study: Online game testing using massive virtual player, in: Control and Automation, and Energy System Engineering, Springer, 2011, pp. 296–301.
  • [18] S. Iftikhar, M. Z. Iqbal, M. U. Khan, W. Mahmood, An automated model based testing approach for platform games, in: Model Driven Engineering Languages and Systems (MODELS), 2015 ACM/IEEE 18th International Conference on, IEEE, 2015, pp. 426–435.
  • [19] M. Ostrowski, S. Aroudj, Automated regression testing within video game development, GSTF Journal on Computing (JoC) 3 (2) (2013) 60. M.R.Taesiri, M.Habibi & M.A.Fazli: A Video Game Testing Method Utilizing Deep Learning (Regular Paper)
  • 33
  • [20] S. Varvaressos, K. Lavoie, A. B. Massé, S. Gaboury, S. Hallé, Automated bug finding in video games: A case study for runtime monitoring, in: Software Testing, Verification and Validation, 2014 IEEE Seventh International Conference on, IEEE, 2014, pp. 143–152.
  • [21] E. Cuervo, A. Wolman, L. P. Cox, K. Lebeck, A. Razeen,
  • S. Saroiu, M. Musuvathi, Kahawai: High-quality mobile gaming using gpu offload, in: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, ACM, 2015, pp. 121–135.
  • [22] Unity - game engine, https://unity3d.com/, [accessed 20December-2016] (2016).
  • [23] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE transactions on image processing 13 (4) (2004) 600–612.
  • [24] R. Mantiuk, K. J. Kim, A. G. Rempel, W. Heidrich, Hdrvdp-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions, in: ACM Transactions on graphics (TOG), Vol. 30.4, ACM, 2011, p. 40.
  • [25] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, The unreasonable effectiveness of deep features as a perceptual metric, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 586–595.
  • [26] G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.
  • [27] Unreal engine technology, https://www.unrealengine.com/, [accessed 20-December2016] (2016).
  • [28] Renderdoc, a stand-alone graphics debugging tool., https://github.com/ baldurk/renderdoc/, [accessed 20December-2017] (2016).
  • [29] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–
  • 778.
  • [30] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A largescale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255.
  • [31] D. Kang, J. Emmons, F. Abuzaid, P. Bailis, M. Zaharia, Noscope: optimizing neural network queries over video at scale, Proceedings of the VLDB Endowment 10 (11) (2017) 1586–1597.
  • [32] uflex - asset store, goo.gl/gpaUJQ, [Online; accessed 18December-2017] (2017). URL goo.gl/gpaUJQ
  • [33] Kinoglitch: Video glitch effects for unity, https://github.com/keijiro/KinoGlitch, [accessed 18December-2017] (2017).
  • [34] Mobile app testing on devices - aws device farm, https://aws.amazon.com/device-farm/, [accessed 26December-2017] (2017).
About Journal
Editorial Board
Guide For Authors
Submit Your Manuscript
Latest Issue
Past Issues
© Copyright - The CSI Journal on Computer Science and Engineering
Produced by Intelligent Information Solutions Center