Skip to content

android : use "ci-android" branch for CI #7341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
May 18, 2024
Merged

android : use "ci-android" branch for CI #7341

merged 4 commits into from
May 18, 2024

Conversation

ggerganov
Copy link
Member

ref #7154

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 17, 2024 •
edited
Loading

📈 llama.cpp server for bench-server-baseline on Standard_NC4as_T4_v3 for phi-2-q4_0: 545 iterations 🚀

Expand details for performance related PR only
  • Concurrent users: 8, duration: 10m
  • HTTP request : avg=8590.34ms p(95)=20570.66ms fails=, finish reason: stop=489 truncated=56
  • Prompt processing (pp): avg=107.45tk/s p(95)=478.51tk/s
  • Token generation (tg): avg=45.89tk/s p(95)=47.33tk/s
  • ggml-org/models/phi-2/ggml-model-q4_0.gguf parallel=8 ctx-size=16384 ngl=33 batch-size=2048 ubatch-size=256 pp=1024 pp+tg=2048 branch=ci-android commit=007f2ece0a3e6ed45b02524c47540384e0fce0bc

prompt_tokens_seconds

More 
---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 545 iterations"
    y-axis "llamacpp:prompt_tokens_seconds"
    x-axis "llamacpp:prompt_tokens_seconds" 1716020894 --> 1716021522
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 289.06, 289.06, 289.06, 289.06, 289.06, 813.51, 813.51, 813.51, 813.51, 813.51, 724.8, 724.8, 724.8, 724.8, 724.8, 750.47, 750.47, 750.47, 750.47, 750.47, 833.56, 833.56, 833.56, 833.56, 833.56, 840.37, 840.37, 840.37, 840.37, 840.37, 831.33, 831.33, 831.33, 831.33, 831.33, 840.08, 840.08, 840.08, 840.08, 840.08, 843.66, 843.66, 843.66, 843.66, 843.66, 860.61, 860.61, 860.61, 860.61, 860.61, 856.39, 856.39, 856.39, 856.39, 856.39, 865.16, 865.16, 865.16, 865.16, 865.16, 871.32, 871.32, 871.32, 871.32, 871.32, 863.6, 863.6, 863.6, 863.6, 863.6, 891.92, 891.92, 891.92, 891.92, 891.92, 897.36, 897.36, 897.36, 897.36, 897.36, 902.48, 902.48, 902.48, 902.48, 902.48, 892.29, 892.29, 892.29, 892.29, 892.29, 898.64, 898.64, 898.64, 898.64, 898.64, 880.61, 880.61, 880.61, 880.61, 880.61, 883.5, 883.5, 883.5, 883.5, 883.5, 881.83, 881.83, 881.83, 881.83, 881.83, 877.35, 877.35, 877.35, 877.35, 877.35, 878.14, 878.14, 878.14, 878.14, 878.14, 867.88, 867.88, 867.88, 867.88, 867.88, 868.19, 868.19, 868.19, 868.19, 868.19, 869.7, 869.7, 869.7, 869.7, 869.7, 861.31, 861.31, 861.31, 861.31, 861.31, 859.6, 859.6, 859.6, 859.6, 859.6, 860.68, 860.68, 860.68, 860.68, 860.68, 859.91, 859.91, 859.91, 859.91, 859.91, 863.48, 863.48, 863.48, 863.48, 863.48, 862.15, 862.15, 862.15, 862.15, 862.15, 864.95, 864.95, 864.95, 864.95, 864.95, 876.05, 876.05, 876.05, 876.05, 876.05, 876.37, 876.37, 876.37, 876.37, 876.37, 858.05, 858.05, 858.05, 858.05, 858.05, 858.08, 858.08, 858.08, 858.08, 858.08, 856.48, 856.48, 856.48, 856.48, 856.48, 853.14, 853.14, 853.14, 853.14, 853.14, 852.95, 852.95, 852.95, 852.95, 852.95, 855.09, 855.09, 855.09, 855.09, 855.09, 855.06, 855.06, 855.06, 855.06, 855.06, 819.38, 819.38, 819.38, 819.38, 819.38, 828.28, 828.28, 828.28, 828.28, 828.28, 828.76, 828.76, 828.76, 828.76, 828.76, 827.09, 827.09, 827.09, 827.09, 827.09, 825.15, 825.15, 825.15, 825.15, 825.15, 830.15, 830.15, 830.15, 830.15, 830.15, 832.33, 832.33, 832.33, 832.33, 832.33, 839.16, 839.16, 839.16, 839.16, 839.16, 838.53, 838.53, 838.53, 838.53, 838.53, 841.19, 841.19, 841.19, 841.19, 841.19, 845.67, 845.67, 845.67, 845.67, 845.67, 844.98, 844.98, 844.98, 844.98, 844.98, 835.2, 835.2, 835.2, 835.2, 835.2, 837.47, 837.47, 837.47, 837.47, 837.47, 837.25, 837.25, 837.25, 837.25, 837.25, 838.44, 838.44, 838.44, 838.44, 838.44, 840.1, 840.1, 840.1, 840.1, 840.1, 844.04, 844.04, 844.04, 844.04, 844.04]
Loading
predicted_tokens_seconds
More 
---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 545 iterations"
    y-axis "llamacpp:predicted_tokens_seconds"
    x-axis "llamacpp:predicted_tokens_seconds" 1716020894 --> 1716021522
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 41.59, 41.59, 41.59, 41.59, 41.59, 30.05, 30.05, 30.05, 30.05, 30.05, 31.36, 31.36, 31.36, 31.36, 31.36, 33.45, 33.45, 33.45, 33.45, 33.45, 33.23, 33.23, 33.23, 33.23, 33.23, 33.36, 33.36, 33.36, 33.36, 33.36, 33.92, 33.92, 33.92, 33.92, 33.92, 34.43, 34.43, 34.43, 34.43, 34.43, 34.13, 34.13, 34.13, 34.13, 34.13, 34.12, 34.12, 34.12, 34.12, 34.12, 33.65, 33.65, 33.65, 33.65, 33.65, 33.65, 33.65, 33.65, 33.65, 33.65, 32.47, 32.47, 32.47, 32.47, 32.47, 32.07, 32.07, 32.07, 32.07, 32.07, 31.29, 31.29, 31.29, 31.29, 31.29, 30.18, 30.18, 30.18, 30.18, 30.18, 30.04, 30.04, 30.04, 30.04, 30.04, 30.31, 30.31, 30.31, 30.31, 30.31, 30.03, 30.03, 30.03, 30.03, 30.03, 30.02, 30.02, 30.02, 30.02, 30.02, 30.0, 30.0, 30.0, 30.0, 30.0, 30.07, 30.07, 30.07, 30.07, 30.07, 30.25, 30.25, 30.25, 30.25, 30.25, 30.15, 30.15, 30.15, 30.15, 30.15, 30.17, 30.17, 30.17, 30.17, 30.17, 30.44, 30.44, 30.44, 30.44, 30.44, 30.48, 30.48, 30.48, 30.48, 30.48, 30.33, 30.33, 30.33, 30.33, 30.33, 30.62, 30.62, 30.62, 30.62, 30.62, 30.7, 30.7, 30.7, 30.7, 30.7, 30.82, 30.82, 30.82, 30.82, 30.82, 30.92, 30.92, 30.92, 30.92, 30.92, 31.09, 31.09, 31.09, 31.09, 31.09, 31.13, 31.13, 31.13, 31.13, 31.13, 31.07, 31.07, 31.07, 31.07, 31.07, 30.85, 30.85, 30.85, 30.85, 30.85, 30.83, 30.83, 30.83, 30.83, 30.83, 30.23, 30.23, 30.23, 30.23, 30.23, 30.18, 30.18, 30.18, 30.18, 30.18, 30.28, 30.28, 30.28, 30.28, 30.28, 30.46, 30.46, 30.46, 30.46, 30.46, 30.49, 30.49, 30.49, 30.49, 30.49, 30.67, 30.67, 30.67, 30.67, 30.67, 30.38, 30.38, 30.38, 30.38, 30.38, 29.99, 29.99, 29.99, 29.99, 29.99, 29.81, 29.81, 29.81, 29.81, 29.81, 28.75, 28.75, 28.75, 28.75, 28.75, 28.73, 28.73, 28.73, 28.73, 28.73, 28.81, 28.81, 28.81, 28.81, 28.81, 28.9, 28.9, 28.9, 28.9, 28.9, 28.94, 28.94, 28.94, 28.94, 28.94, 29.03, 29.03, 29.03, 29.03, 29.03, 29.07, 29.07, 29.07, 29.07, 29.07, 28.98, 28.98, 28.98, 28.98, 28.98, 29.0, 29.0, 29.0, 29.0, 29.0, 29.07, 29.07, 29.07, 29.07, 29.07, 29.21, 29.21, 29.21, 29.21, 29.21, 29.38, 29.38, 29.38, 29.38, 29.38, 29.44, 29.44, 29.44, 29.44, 29.44, 29.54, 29.54, 29.54, 29.54, 29.54]
Loading

Details

kv_cache_usage_ratio

More 
---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 545 iterations"
    y-axis "llamacpp:kv_cache_usage_ratio"
    x-axis "llamacpp:kv_cache_usage_ratio" 1716020894 --> 1716021522
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.05, 0.05, 0.05, 0.05, 0.05, 0.34, 0.34, 0.34, 0.34, 0.34, 0.29, 0.29, 0.29, 0.29, 0.29, 0.13, 0.13, 0.13, 0.13, 0.13, 0.21, 0.21, 0.21, 0.21, 0.21, 0.18, 0.18, 0.18, 0.18, 0.18, 0.09, 0.09, 0.09, 0.09, 0.09, 0.14, 0.14, 0.14, 0.14, 0.14, 0.22, 0.22, 0.22, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.21, 0.25, 0.25, 0.25, 0.25, 0.25, 0.16, 0.16, 0.16, 0.16, 0.16, 0.32, 0.32, 0.32, 0.32, 0.32, 0.18, 0.18, 0.18, 0.18, 0.18, 0.34, 0.34, 0.34, 0.34, 0.34, 0.26, 0.26, 0.26, 0.26, 0.26, 0.2, 0.2, 0.2, 0.2, 0.2, 0.12, 0.12, 0.12, 0.12, 0.12, 0.16, 0.16, 0.16, 0.16, 0.16, 0.19, 0.19, 0.19, 0.19, 0.19, 0.3, 0.3, 0.3, 0.3, 0.3, 0.21, 0.21, 0.21, 0.21, 0.21, 0.13, 0.13, 0.13, 0.13, 0.13, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.17, 0.13, 0.13, 0.13, 0.13, 0.13, 0.17, 0.17, 0.17, 0.17, 0.17, 0.33, 0.33, 0.33, 0.33, 0.33, 0.1, 0.1, 0.1, 0.1, 0.1, 0.11, 0.11, 0.11, 0.11, 0.11, 0.19, 0.19, 0.19, 0.19, 0.19, 0.16, 0.16, 0.16, 0.16, 0.16, 0.17, 0.17, 0.17, 0.17, 0.17, 0.14, 0.14, 0.14, 0.14, 0.14, 0.22, 0.22, 0.22, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.21, 0.38, 0.38, 0.38, 0.38, 0.38, 0.41, 0.41, 0.41, 0.41, 0.41, 0.37, 0.37, 0.37, 0.37, 0.37, 0.11, 0.11, 0.11, 0.11, 0.11, 0.16, 0.16, 0.16, 0.16, 0.16, 0.12, 0.12, 0.12, 0.12, 0.12, 0.15, 0.15, 0.15, 0.15, 0.15, 0.33, 0.33, 0.33, 0.33, 0.33, 0.54, 0.54, 0.54, 0.54, 0.54, 0.59, 0.59, 0.59, 0.59, 0.59, 0.51, 0.51, 0.51, 0.51, 0.51, 0.23, 0.23, 0.23, 0.23, 0.23, 0.19, 0.19, 0.19, 0.19, 0.19, 0.13, 0.13, 0.13, 0.13, 0.13, 0.21, 0.21, 0.21, 0.21, 0.21, 0.12, 0.12, 0.12, 0.12, 0.12, 0.15, 0.15, 0.15, 0.15, 0.15, 0.32, 0.32, 0.32, 0.32, 0.32, 0.19, 0.19, 0.19, 0.19, 0.19, 0.16, 0.16, 0.16, 0.16, 0.16, 0.14, 0.14, 0.14, 0.14, 0.14, 0.12, 0.12, 0.12, 0.12, 0.12, 0.15, 0.15, 0.15, 0.15, 0.15, 0.16, 0.16, 0.16, 0.16, 0.16, 0.18, 0.18, 0.18, 0.18, 0.18]
Loading
requests_processing
More 
---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 545 iterations"
    y-axis "llamacpp:requests_processing"
    x-axis "llamacpp:requests_processing" 1716020894 --> 1716021522
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.0, 5.0, 5.0, 5.0, 5.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 4.0, 4.0, 4.0, 4.0, 4.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 6.0, 6.0, 6.0, 6.0, 6.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 4.0, 4.0, 4.0, 4.0, 4.0, 6.0, 6.0, 6.0, 6.0, 6.0, 4.0, 4.0, 4.0, 4.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 4.0, 4.0, 4.0, 4.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 7.0, 7.0, 7.0, 7.0, 7.0, 4.0, 4.0, 4.0, 4.0, 4.0, 6.0, 6.0, 6.0, 6.0, 6.0, 2.0, 2.0, 2.0, 2.0, 2.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 5.0, 5.0, 5.0, 5.0, 7.0, 7.0, 7.0, 7.0, 7.0, 5.0, 5.0, 5.0, 5.0, 5.0, 7.0, 7.0, 7.0, 7.0, 7.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 2.0, 2.0, 2.0, 2.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 2.0, 6.0, 6.0, 6.0, 6.0, 6.0, 4.0, 4.0, 4.0, 4.0, 4.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 4.0, 4.0, 4.0, 4.0, 4.0, 1.0, 1.0, 1.0, 1.0, 1.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 1.0, 1.0, 1.0, 1.0, 1.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0]
Loading

@ggerganov ggerganov marked this pull request as ready for review May 17, 2024 13:23
@slaren
Copy link
Member

slaren commented May 17, 2024

I think the FetchContent can be replaced with an add_subdirectory to the root directory of the repository, then it will use the llama.cpp code in the PR. The FetchContent stuff can be left commented as an example for other applications.

@mofosyne mofosyne added android Issues specific to Android Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix devops improvements to build systems and github actions labels May 17, 2024
@mofosyne mofosyne added high priority Very important issue merge ready indicates that this may be ready to merge soon and is just holding out in case of objections labels May 18, 2024
@mofosyne mofosyne self-requested a review May 18, 2024 10:26
@mofosyne mofosyne merged commit 511182e into master May 18, 2024
72 checks passed
ggerganov added a commit that referenced this pull request May 19, 2024
@ggerganov
cmake : update android comments (#7341)
854d365
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mofosyne mofosyne mofosyne approved these changes

Assignees
No one assigned
Labels
android Issues specific to Android devops improvements to build systems and github actions examples high priority Very important issue merge ready indicates that this may be ready to merge soon and is just holding out in case of objections Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ggerganov @slaren @mofosyne