[llvm-lit] Lit's built-in diff command causing clang test to fail when run with internal shell

[connieyzhu]

Lit's built-in diff command gives the incorrect output in clang/test/APINotes/yaml-roundtrip.test, causing the test to fail in lit's internal shell.

Error message:

FAIL: Clang :: APINotes/yaml-roundtrip.test (2 of 2)
******************** TEST 'Clang :: APINotes/yaml-roundtrip.test' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
/usr/local/google/home/connieyzhu/llvm-fork/build/bin/apinotes-test /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes > /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# executed command: /usr/local/google/home/connieyzhu/llvm-fork/build/bin/apinotes-test /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# RUN: at line 2
not diff -bu /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result > /tmp/diff
# executed command: not diff -bu /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# RUN: at line 3
not diff -b /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result | /usr/local/google/home/connieyzhu/llvm-fork/build/bin/FileCheck /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test
# executed command: not diff -b /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# executed command: /usr/local/google/home/connieyzhu/llvm-fork/build/bin/FileCheck /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test
# .---command stderr------------
# | /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test:12:8: error: CHECK: expected string not found in input
# | CHECK: 7c8
# |        ^
# | <stdin>:1:1: note: scanning from here
# | *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# | ^
# | <stdin>:1:11: note: possible intended match here
# | *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# |           ^
# | 
# | Input file: <stdin>
# | Check file: /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             1: *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes 
# | check:12'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
# | check:12'1               ?                                                                                                                     possible intended match
# |             2: --- /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result 
# | check:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |             3: *************** 
# | check:12'0     ~~~~~~~~~~~~~~~~
# |             4: *** 1,28 **** 
# | check:12'0     ~~~~~~~~~~~~~~
# |             5:  Name: SimpleKit 
# | check:12'0     ~~~~~~~~~~~~~~~~~
# |             6:  Classes: 
# | check:12'0     ~~~~~~~~~~
# |             .
# |             .
# |             .
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

Steps to reproduce:

1. CMake Config:
   cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang;llvm;clang-tools-extra;lld;lldb;bolt;mlir;libclc;polly" -DLLVM_ENABLE_RUNTIMES="compiler-rt;libc;libcxx;libunwind;libcxxabi" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -B build -S .
2. Use this command to run lit's internal shell on this test.
   LIT_USE_INTERNAL_SHELL=1 LIT_FILTER=yaml-roundtrip.test ninja check-clang

Lit's built-in diff currently defaults to context diff (diff -c), while the expected output uses regular Unix diff.