Backtesting result calculation is weird

[h4l-yup]

Describe the bug

poetry run python src/backtester.py --ticker TSLA --start-date 2024-12-01 --end-date 2024-12-20
This was my command and below is the result.

PORTFOLIO SUMMARY:
Cash Balance: $311,915.44
Total Position Value: $-1,807.86
Total Value: $310,107.58
Return: +210.11%
Sharpe Ratio: 11.04
Sortino Ratio: 142.50
Max Drawdown: -2.38%



+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| Date       | Ticker   |  Action  |   Quantity |   Price |   Shares |   Position Value |   Bullish |   Bearish |   Neutral |
+============+==========+==========+============+=========+==========+==================+===========+===========+===========+
| 2024-12-02 | TSLA     |  SHORT   |         57 |  357.09 |      -57 |       -20,354.13 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-03 | TSLA     |  SHORT   |         10 |  357.09 |      -67 |       -23,925.03 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-04 | TSLA     |   HOLD   |          0 |  351.42 |      -67 |       -23,545.14 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-05 | TSLA     |  SHORT   |          2 |  357.93 |      -69 |       -24,697.17 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-06 | TSLA     |   HOLD   |          0 |  369.49 |      -69 |       -25,494.81 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-09 | TSLA     |   HOLD   |          0 |  389.79 |      -69 |       -26,895.51 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-10 | TSLA     |   HOLD   |          0 |  389.79 |      -69 |       -26,895.51 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-11 | TSLA     |   HOLD   |          0 |  400.99 |      -69 |       -27,668.31 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-12 | TSLA     |  SHORT   |         58 |  424.77 |     -127 |       -53,945.79 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-13 | TSLA     |   HOLD   |          0 |   418.1 |     -127 |       -53,098.70 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-16 | TSLA     |  SHORT   |         68 |  463.02 |     -195 |       -90,288.90 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-17 | TSLA     |  SHORT   |         78 |  463.02 |     -273 |      -126,404.46 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-18 | TSLA     |   HOLD   |          0 |  479.86 |     -273 |      -131,001.78 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-19 | TSLA     |  SHORT   |         98 |  440.13 |     -371 |      -163,288.23 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-12-20 | TSLA     |  SHORT   |        119 |  436.17 |     -490 |      -213,723.30 |         0 |         1 |         0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+






PORTFOLIO PERFORMANCE SUMMARY:
Total Return: 210.11%
Total Realized Gains/Losses: $0.00
2025-03-31 00:09:36.589 python[41723:80655128] +[IMKClient subclass]: chose IMKClient_Modern
2025-03-31 00:09:36.589 python[41723:80655128] +[IMKInputSession subclass]: chose IMKInputSession_Modern

Sharpe Ratio: 11.58
Maximum Drawdown: -2.38% (on 2024-12-11)
Win Rate: 66.67%
Win/Loss Ratio: 11.22
Max Consecutive Wins: 4
Max Consecutive Losses: 4

As you can see, the price of Tesla just goes up and the ai made all the decisions of SHORT.
Therefore it can not make profits at all but the result says total value goes up.

[virattt]

Hey @h4l-yup - which agent(s) and LLM did you use?

[dramfin]

I am running into the same issue with the AI signal "flip-flopping." I was wondering if there is a way to make decisions more deterministic and implement a delta to buffer the flip-flopping and reinforce conviction.

[h4l-yup]

@virattt
I just used Warren Buffet only.

But when I used Cathie Wood agent, it seems it has similar issues.

[zzzzz1st]

Same issue here. I used Technical Analyst, Fundamentals Analyst, Sentiment Analyst and Valuation Analyst together with the deepseekR1 model. This is the command that I used :
poetry run python src/backtester.py --ticker AAPL --start-date 2024-05-02 --end-date 2024-05-10

And this is the result :

+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| Date | Ticker | Action | Quantity | Price | Shares | Position Value | Bullish | Bearish | Neutral |
+============+==========+==========+============+=========+==========+==================+===========+===========+===========+
| 2024-05-02 | AAPL | SHORT | 118 | 169.3 | -118 | -19,977.40 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-03 | AAPL | SHORT | 20 | 173.03 | -138 | -23,878.14 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-06 | AAPL | HOLD | 0 | 181.71 | -138 | -25,075.98 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-07 | AAPL | SHORT | 135 | 181.71 | -273 | -49,606.83 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-08 | AAPL | SHORT | 162 | 182.4 | -435 | -79,344.00 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-09 | AAPL | SHORT | 194 | 182.74 | -629 | -114,943.46 | 0 | 3 | 1 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+
| 2024-05-10 | AAPL | HOLD | 0 | 184.57 | -629 | -116,094.53 | 1 | 3 | 0 |
+------------+----------+----------+------------+---------+----------+------------------+-----------+-----------+-----------+

PORTFOLIO PERFORMANCE SUMMARY:
Total Return: 109.84%
Total Realized Gains/Losses: $0.00
2025-04-01 20:30:30.320 Python[60644:585465] +[IMKClient subclass]: chose IMKClient_Modern
2025-04-01 20:30:30.320 Python[60644:585465] +[IMKInputSession subclass]: chose IMKInputSession_Modern

Sharpe Ratio: 15.13
Maximum Drawdown: 0.97% (on 2024-05-06)
Win Rate: 71.43%
Win/Loss Ratio: 21.80
Max Consecutive Wins: 3
Max Consecutive Losses: 1

[zzzzz1st]

I just made a pull request that could fix the issue. Could you please review it ? Thank you