metal support for cocos2d-x (#19305)

* remove deprecated files

* remove some deprecated codes

* remove more deprecated codes

* remove ui deprecated codes

* remove more deprecated codes

* remove deprecated codes in ccmenuitem

* remove more deprecated codes in ui

* remove more deprecated codes in ui

* remove more deprecated codes in ui

* remove more deprecated codes

* remove more deprecated codes

* remove more deprecated codes

* remove vr related codes and ignore some modules

* remove allocator

* remove some config

* 【Feature】add back-end project file

* [Feature] add back-end file

* add pipeline descriptor and shader cache

* [Feature] support sprite for backend

* [Feature] remove unneeded code

* [Feature] according to es2.0 spec, you must use clamp-to-edge as  texture wrap mode, and no mipmapping for non-power-of-two texture

* [Feature] set texture wrap mode to clamp-to-edge, and no mipmapping for non-power-of-two texture

* [Feature] remove macro define to .cpp file

* [Feature] add log info

* [Feature] add PipelineDescriptor for TriangleCommand

* [Feature] add PipelineDescriptor object as member of TriangleCommand

* [Feature] add getPipelineDescriptor method

* add renderbackend

* complete pipeline descriptor

* [Feature] add viewport in RenderCommand

* set viewport when rendrering

* [Feature] occur error when using RendererBackend, to be fixed.

* a workaround to fix black screen on macOS 10.14 (#19090)

* add rendererbackend init function

* fix typo

* [Feature] modify testFile

* [BugFix] modify shader path

* [Feature] set default viewport

* fix projection

* [Feature] modify log info

* [BugFix] change viewport data type to int

* [BugFix] add BindGroup to PipelienDescriptor

* [BugFix] change a_position to vec3 in sprite.vert

* [BugFix] set vertexLayout according to V3F_C4B_T2F structure

* [Feature] revert a_position to vec4

* [Feature] renderer should not use gl codes directly

* [Feature] it's better not use default value parameter

* fix depth test setting

* rendererbackend -> renderer

* clear color and depth at begin

* add metal backend

* metal support normalized attribute

* simplify codes

* update external

* add render pass desctriptor in pipeline descriptor

* fix warnings

* fix crash and memeory leak

* refactor Texture2D

* put pipeline descriptor into render command

* simplify codes

* [Feature] update Sprite

* fix crash when closing app

* [Feature] update SpriteBatchNode and TextureAtlas

* support render texture(not finish)

* [Feature] remove unused code

* make tests work on mac

* fix download-deps path error

* make tests work on iOS

* [Feature] support ttf under normal label effect

* refactor triangle command processing

* let renderer handle more common commands

* refactor backend

* make render texture work

* [Feature] refactor backend for GL

* [Feature]Renaming to make it easy to understand

* [Feature] change warp mode to CLAMP_TO_EDGE

* fix ghost

* simplify visit render queue logic

* support progress timer without rial mode

* support partcile system

* Feature/update label (#149)

* [BugFix] fix compile error

* [Feature] support outline effect in ios

* [Feature] add shader file

* [BugFix] fix begin and end RenderPass

* [Feature] update CustomCommand

* [Feature] revert project.pbxproj

* [Feature] simplify codes

* [BugFix] pack AI88 to RGBA8888 only when outline enable

* [Feature] support shadow effect in Label

* [Feature] support BMFont

* [Feature] support glow effect

* [Feature] simplify shader files

* LabelAtlas work

* handle blend function correctly

* support tile map

* don't share buffer in metal

* alloc buffer size as needed

* support more tilemap

* Merge branch 'minggo/metal-support' into feature/updateLabel

* minggo/metal-support:
  support tile map
  handle blend function correctly
  LabelAtlas work
  Feature/update label (#149)
  support partcile system

# Conflicts:
#	cocos/2d/CCLabel.cpp
#	cocos/2d/CCSprite.cpp
#	cocos/2d/CCSpriteBatchNode.cpp
#	cocos/renderer/CCQuadCommand.cpp
#	cocos/renderer/CCQuadCommand.h

* render texture work without saving file

* use global viewport

* grid3d works

* remove grabber

* tiled3d works

* [BugFix] fix label bug

* [Feature] add updateSubData for buffer

* [Feature] remove setVertexCount

* support depth test

* add callback command

* [Feature] add UITest

* [Feature] update UITest

* [Feature] remove unneeded codes

* fix custom command issue

* fix layer color blend issue

* [BugFix] fix iOS compile error

* [Feature] remove unneeded codes

* [Feature] fix updateVertexBuffer

* layerradial works

* add draw test back

* fix batch issue

* fix compiling error

* [BugFix] support ETC1

* [BugFix] get the correct pipelineDescriptor

* [BugFix]  skip draw when backendTexture nullptr

* clipping node support

* [Feature] add shader files

* fix stencil issue in metal

* [Feature] update UILayoutTest

* [BugFix] skip drawing when vertexCount is zero

* refactor renderer

* add set global z order for stencil manager commands

* fix warnings caused by type

* remove viewport in render command

* [Feature] fix warnings caused by type

* [BugFix] clear vertexCount and indexCount for CustomComand when needed

* [Feature] update clear for CustomCommand

* ios use metal

* fix viewport issue

* fix LayerColorGradient crash

* [cmake] transport to android and windows (#160)

* save point 1

* compile on windows

* run on android

* revert useless change

* android set CC_ENABLE_CACHE_TEXTURE_DATA to 1

* add initGlew

* fix android crash

* add TODO new-renderer

* review update

* revert onGLFWWindowPosCallback

* fix android compiling error

* Impl progress radial (#162)

* progresstimer add radial impl

* default drawType to element

* dec invoke times of createVertexBuffer (#163)

* support depth/stencil format for gl backend

* simplify progress timer codes

* support motionstreak, effect is wrong

* fix motionstreak issue

* [Feature] update Scissor Test (#161)

* [Feature] update Scissor Test

* [Feature] update ScissorTest

* [Feature] rename function

* [Feature] get constant reference if needed

* [Feature] show render status (#164)

* improve performance

* fix depth state

* fill error that triangle vertex/index number bigger than buffer

* fix compiline error in release mode

* fix buffer conflict between CPU and GPU on iOS/macOS

* Renderer refactor (#165)

* use one vertes/index buffer with opengl

* fix error on windows

* custom command support index format config

* CCLayer: compact vertex data structure

* update comment

* fix doc

* support fast tilemap

* pass index format instead

* fix some wrong effect

* fix render texture error

* fix texture per-element size

* fix texture format error

* BlendFunc type refactor, GLenum -> backend::BlendFactor  (#167)

* BlendFunc use backend::BlendFactor as inner field

* update comments

* use int to replace GLenum

* update xcode project fiel

* rename to GLBlendConst

* add ccConstants.h

* update xcode project file

* update copyright

* remove primitive command

* remove CCPrimitive.cpp/.h

* remove deprecated files

* remove unneeded files

* remove multiple view support

* remove multiple view support

* remove the usage of frame buffer in camera

* director don't use frame buffer

* remove FrameBuffer

* remove BatchCommand

* add some api reference

* add physics2d back

* fix crash when close app on mac

* improve render texture

* fix rendertexture issue

* fix rendertexture issue

* simplify codes

* CMake support for mac & ios (#169)

* update cmake

* fix compile error

* update 3rd libs version

* remove CCThread.h/.cpp

* remove ccthread

* use audio engine to implement simple audio engine

* remove unneeded codes

* remove deprecated codes

* remove winrt macro

* remove CC_USE_WIC

* set partcile blend function in more elegant way

* remove unneeded codes

* remove unneeded codes

* cmake works on windows

* update project setting

* improve performance

* GLFloat -> float

* sync v3 cmake improvements into metal-support (#172)

* pick: modern cmake, compile definitions improvement (#19139)

* modern cmake, use target_compile_definitions partly

* simplify macro define, remove USE_*

* modern cmake, macro define

* add physics 2d macro define into ccConfig.h

* remove USE_CHIPMUNK macro in build.gradle

* remove CocosSelectModule.cmake

* shrink useless define

* simplify compile options config, re-add if necessary

* update external for tmp CI test

* un-quote target_compile_options value

* add "-g" parameter only when debug mode

* keep single build type when generator Xcode & VS projecy

* update external for tmp CI tes

* add static_cast<char>(-1), fix -Wc++11-narrowing

* simplify win32 compile define

* not modify code, only improve compile options

# Conflicts:
#	.gitignore
#	cmake/Modules/CocosConfigDepend.cmake
#	cocos/CMakeLists.txt
#	external/config.json
#	tests/cpp-tests/CMakeLists.txt

* modern cmake, improve cmake_compiler_flags (#19145)

* cmake_compiler_flags

* Fix typo

* Fix typo2

* Remove chanages from Android.mk

*  correct lua template cmake build (#19149)

* don't add -Wno-deprecated into jsb target

* correct lua template cmake build

* fix win32 lua template compile error

* prevent cmake in-source-build friendly (#19151)

* pick: Copy resources to "Resources/" on win32 like in linux configuration

* add "/Z7" for cpp-tests on windows

* [cmake] fix iOS xcode property setting failed (#19208)

* fix iOS xcode property setting failed

* use search_depend_libs_recursive at dlls collect

* fix typo

* [cmake] add find_host_library into iOS toolchain file (#19230)

* pick: [lua android] use luajit & template cmake update (#19239)

* increase cmake stability ,  remove tests/CMakeLists.txt (#19261)

* cmake win32 Precompiled header (#19273)

* Precompiled header

* Fix

* Precompiled header for cocos

* Precompiled header jscocos2d

* Fix for COCOS2D_DEBUG is always 1 on Android (#19291)

Related #19289

* little build fix, tests cpp-tests works on mac

* sync v3 build related codes into metal-support (#173)

* strict initialization for std::array

* remove proj.win32 project configs

* modern cmake, cmake_cleanup_remove_unused_variables (#19146)

* Switch travis CI to xenial (#19207)

* Switch travis CI to xenial

* Remove language: android

* Set language: cpp

* Fix java problem

* Update sdkmanager

* Fix sdkmanger

* next sdkmanager fix

* Remove xenial from android

* revert to sdk-tools-{system}-3859397

* Remove linux cmake install

* Update before-install.sh

* Update .travis.yml

* Simplify install-deps-linux.sh, tested on Ubuntu 16.04 (#19212)

* Simplify install-deps-linux.sh

* Cleanup

* pick: install ninja

* update cocos2d-console submodule

* for metal-support alpha release, we only test cpp

* add HelloCpp into project(Cocos2d-x) for tmp test

* update extenal metal-support-4

* update uniform setting

* [Feature] update BindGroup

* [Feature] empty-test

* [Feature] cpp-test

* [Feature] fix GL compiler error

* [Feature] fix GL crash

* [Feature] empty-test

* [Feature] cpp-tests

* [feature] improve frameRate

* [feature] fix opengl compile error

* [feature] fix opengl compile error

* [BugFix] fix compute maxLocation error

* [Feature] update setting unifrom

* [Feature] fix namespace

* [Feature] remove unneeded code

* [Bugfix] fix project file

* [Feature] update review

* [texture2d] impl texture format support  (#175)

* texture update

* update

* update texture

* commit

* compile on windows

* ddd

* rename

* rename methods

* no crash

* save gl

* save

* save

* rename

* move out pixel format convert functions

* metal crash

* update

* update android

* support gles compressed texture format

* support more compress format

* add more conversion methods

* ss

* save

* update conversion methods

* add PVRTC format support

* reformat

* add marco linux

* fix GL marcro

* pvrtc supported only by ios 8.0+

* remove unused cmake

* revert change

* refactor Texture2D::initWithData

* fix conversion log

* refactor Texture2D::initWithData

* remove some OpenGL constants for PVRTC

* add todo

* fix typo

* AutoTest works on mac/iOS by disable part cases, sync v3 bug fix (#174)

* review cpp-tests, and fix part issues on start auto test

* sync png format fix: Node:Particle3D abnormal texture effects #19204

* fix cpp-tests SpritePolygon crash, wrong png format (#19170)

* fix wrong png convert format from sRGB to Gray

* erase plist index if all frames was erased

* test_A8.png have I8 format, fix it

* [CCSpriteCache] allow re-add plist & add testcase (#19175)

* allow re-add plist & add testcase

* remove comments/rename method/update testcase

* fix isSpriteFramesWithFileLoaded & add testcase

* remove used variable

* remove unused variable

* fix double free issues when js/lua-tests exit on iOS (#19236)

* disable part cases, AutoTest works without crash on mac

* update cocos2dx files json, to test cocos new next

* fix spritecache plist parsing issue (#19269)

* [linux] Fix FileUtils::getContents with folder (#19157)

* fix FileUtils::getContents on linux/mac

* use stat.st_mode

* simplify

* [CCFileUtils] win32 getFileSize (#19176)

* win32 getFileSize

* fix stat

* [cpp test-Android]20：FileUtils/2 change title (#19197)

* sync #19200

* sync #19231

* [android lua] improve performance of lua loader (#19234)

* [lua] improve performance of lua loader

* remove cache fix

* Revert "fix spritecache plist parsing issue (#19269)"

This reverts commit f3a85ec.

* remove win32 project files ref in template.json

* add metal framework lnk ref into cpp template

* test on iOS, and disable part cases

* alBufferData instead of alBufferDataStatic for small audio file on Apple (#19227)

* changes AudioCache to use alBufferData instead of alBufferDataStatic

(also makes test 19 faster to trigger openal bugs faster)

The original problem: CrashIfClientProvidedBogusAudioBufferList
#18948
is not happening anymore, but there's still a not very frequent issue
that makes OpenAL crash with a call stack like this.
AudioCache::readDataTask > alBufferData > CleanUpDeadBufferList

It happes more frequently when the device is "cold", which means after
half an hour of not using the device (locked).

I could not find the actual source code for iOS OpenAL, so I used the
macOS versions:
https://opensource.apple.com/source/OpenAL/OpenAL-48.7/Source/OpenAL/oalImp.cpp.auto.html

They seem to use CAGuard.h to make sure the dead buffer list
has no threading issues. I'm worried because the CAGuard code I found
has macos and win32 define but no iOS, so I'm not sure. I guess the
iOS version is different and has the guard.

I could not find a place in the code that's unprotected by the locks
except the InitializeBufferMap() which should not be called more than
once from cocos, and there's a workaround in AudioEngine-impl for it.

I reduced the occurence of the CleanUpDeadBufferList crash by moving
the guard in ~AudioCache to cover the alDeleteBuffers call.

* remove hack method "setTimeout" on audio

* AutoTest works on iOS

* support set ios deployment target for root project

* enable all texture2d cases, since Jiang have fixed

* add CCTextureUtils to xcode project file (#176)

* add leak cases for SpriteFrameCache (#177)

* re-add SpriteFrameCache cases

* update template file json

* Update SpriteFrameCacheTest.cpp

* fix compiling error

Author: minggo

.appveyor.yml

@@ -37,6 +37,7 @@ branches:
   - v3-doc
   - v3.11_backup
   - v35-for-tizen
+  - metal-support
 
 clone_depth: 1
 

.gitignore

@@ -168,3 +168,7 @@ tests/lua-tests/project/proj.tizen/res/
 cmake-build-*/
 .vs
 cmake_build*
+metal-support*.zip
+**/.project
+
+desktop.ini

.travis.yml

@@ -2,6 +2,7 @@ matrix:
   include:
     # linux
     - os: linux
+      dist: xenial
       env: 
         - BUILD_TARGET=linux 
         - GEN_BINDING_AND_COCOSFILE=true
@@ -61,9 +62,10 @@ matrix:
       sudo: required
     - os: linux
       env: BUILD_TARGET=android_cocos_new_test
-      language: cpp
+      language: android
       sudo: required
     - os: linux
+      dist: xenial
       env: BUILD_TARGET=linux_cocos_new_test
       language: cpp
       sudo: required

CMakeLists.txt

@@ -34,15 +34,34 @@ project(Cocos2d-x)
 set(COCOS2DX_ROOT_PATH ${CMAKE_CURRENT_SOURCE_DIR})
 set(CMAKE_MODULE_PATH ${COCOS2DX_ROOT_PATH}/cmake/Modules/)
 
+# prevent in-source-build
+include(PreventInSourceBuilds)
+AssureOutOfSourceBuilds()
+
 # works before build libcocos2d
 include(CocosBuildSet)
 
-# default tests include lua, js test project, so we set those option on to build libs
-set(BUILD_LUA_LIBS ON)
-set(BUILD_JS_LIBS ON)
+# for metal-support alpha release, we only test cpp
+set(BUILD_LUA_LIBS OFF) 
+set(BUILD_JS_LIBS OFF)
+
+add_subdirectory(${COCOS2DX_ROOT_PATH}/cocos ${ENGINE_BINARY_PATH}/cocos/core)
+
+# prevent tests project to build "cocos2d-x/cocos" again
+set(BUILD_ENGINE_DONE ON)
+
+# add cpp tests default
+add_subdirectory(${COCOS2DX_ROOT_PATH}/tests/cpp-empty-test ${ENGINE_BINARY_PATH}/tests/cpp-empty-test)
+add_subdirectory(${COCOS2DX_ROOT_PATH}/tests/cpp-tests ${ENGINE_BINARY_PATH}/tests/cpp-tests)
+
+if(BUILD_LUA_LIBS)
+  add_subdirectory(${COCOS2DX_ROOT_PATH}/tests/lua-empty-test/project ${ENGINE_BINARY_PATH}/tests/lua-empty-test)
+  add_subdirectory(${COCOS2DX_ROOT_PATH}/tests/lua-tests/project ${ENGINE_BINARY_PATH}/tests/lua-test)
+endif(BUILD_LUA_LIBS)
 
-if(NOT USE_COCOS_PREBUILT)
-    add_subdirectory(${COCOS2DX_ROOT_PATH}/cocos ${ENGINE_BINARY_PATH}/cocos/core)
+if(BUILD_JS_LIBS)
+  add_subdirectory(${COCOS2DX_ROOT_PATH}/tests/js-tests/project ${ENGINE_BINARY_PATH}/tests/js-tests)
 endif()
-add_subdirectory(${COCOS2DX_ROOT_PATH}/tests ${ENGINE_BINARY_PATH}/tests)
 
+# add cpp-template-default into project(Cocos2d-x) for tmp test
+add_subdirectory(${COCOS2DX_ROOT_PATH}/templates/cpp-template-default ${ENGINE_BINARY_PATH}/tests/HelloCpp)