Xlsx Writer Unhides Explicitly Hidden Row in Filter Range - Minor Breaking Change (#2414)

Fix #1641. Excel allows explicit hiding of row after filter is applied, but PhpSpreadsheet automatically invokes showHideRows on all auto-filters, preventing users from doing the same. Change to invoke showHideRows only if it hasn't already been invoked, or if filter criteria have changed since it was last invoked. Autofilters read in from an existing spreadsheet are assumed to be already invoked.

This is potentially a breaking change, probably a minor one. The conditions to set up 1641 are probably uncommon, but users who meet those conditions and are happy with the current behavior will see a break. The new behavior is closer to how Excel itself behaves. A new method `reevaluateAutoFilters` is added to `Spreadsheet`; this can be used to restore the old behavior if desired. The new method is added to the documentation, along with a description of how the situation described in 1641 is handled in Excel and PhpSpreadsheet.

While examining Excel's behavior, it became evident that, although a filter is applied to an entire column, it is actually applied only to the rows that are populated when the filter is defined, as can be verified by examining the XML definition of the filter. When you re-apply the filter, rows that have been added since are considered. It would be useful to provide PhpSpreadsheet with a method to do the same. I have added, and documented, `setRangeToMaxRow` to `AutoFilter`.

Author: oleibman

docs/topics/autofilters.md

@@ -65,6 +65,13 @@ $spreadsheet->getActiveSheet()->setAutoFilter(
 
 This enables filtering, but does not actually apply any filters.
 
+After setting the range, you can change it so that the end row is the
+last row used on the worksheet:
+
+```php
+$spreadsheet->getActiveSheet()->getAutoFilter()->setRangeToMaxRow();
+```
+
 ## Autofilter Expressions
 
 PHPEXcel 1.7.8 introduced the ability to actually create, read and write
@@ -503,6 +510,23 @@ $autoFilter->showHideRows();
 This will set all rows that match the filter criteria to visible, while
 hiding all other rows within the autofilter area.
 
+Excel allows you to explicitly hide a row after applying a filter even
+if the row wasn't hidden by the filter. However, if a row is hidden *before*
+applying the filter, and the filter is applied, the row will no longer be hidden.
+This can make a difference during PhpSpreadsheet save, since PhpSpreadsheet
+will apply the filter during save if it hasn't been previously applied,
+or if the filter criteria have changed since it was last applied.
+Note that an autofilter read in from an existing spreadsheet is assumed to have been applied.
+Also note that changing the data in the columns being filtered
+does not result in reevaluation in either Excel or PhpSpreadsheet.
+If you wish to re-apply all filters in the spreadsheet
+(possibly just before save):
+```php
+$spreadsheet->reevaluateAutoFilters(false);
+```
+You can specify `true` rather than `false` to adjust the filter ranges
+on each sheet so that they end at the last row used on the sheet.
+
 ### Displaying Filtered Rows
 
 Simply looping through the rows in an autofilter area will still access

src/PhpSpreadsheet/Reader/Xlsx/AutoFilter.php

@@ -61,6 +61,7 @@ private function readAutoFilter($autoFilterRange, $xmlSheet): void
             //    Check for dynamic filters
             $this->readTopTenAutoFilter($filterColumn, $column);
         }
+        $autoFilter->setEvaluated(true);
     }
 
     private function readDateRangeAutoFilter(SimpleXMLElement $filters, Column $column): void

src/PhpSpreadsheet/Spreadsheet.php

@@ -1589,4 +1589,17 @@ public function setTabRatio($tabRatio): void
             throw new Exception('Tab ratio must be between 0 and 1000.');
         }
     }
+
+    public function reevaluateAutoFilters(bool $resetToMax): void
+    {
+        foreach ($this->workSheetCollection as $sheet) {
+            $filter = $sheet->getAutoFilter();
+            if (!empty($filter->getRange())) {
+                if ($resetToMax) {
+                    $filter->setRangeToMaxRow();
+                }
+                $filter->showHideRows();
+            }
+        }
+    }
 }

src/PhpSpreadsheet/Worksheet/AutoFilter.php

@@ -35,6 +35,19 @@ class AutoFilter
      */
     private $columns = [];
 
+    /** @var bool */
+    private $evaluated = false;
+
+    public function getEvaluated(): bool
+    {
+        return $this->evaluated;
+    }
+
+    public function setEvaluated(bool $value): void
+    {
+        $this->evaluated = $value;
+    }
+
     /**
      * Create a new AutoFilter.
      *
@@ -63,6 +76,7 @@ public function getParent()
      */
     public function setParent(?Worksheet $worksheet = null)
     {
+        $this->evaluated = false;
         $this->workSheet = $worksheet;
 
         return $this;
@@ -87,6 +101,7 @@ public function getRange()
      */
     public function setRange($range)
     {
+        $this->evaluated = false;
         // extract coordinate
         [$worksheet, $range] = Worksheet::extractSheetTitle($range, true);
         if (empty($range)) {
@@ -114,6 +129,20 @@ public function setRange($range)
         return $this;
     }
 
+    public function setRangeToMaxRow(): self
+    {
+        $this->evaluated = false;
+        if ($this->workSheet !== null) {
+            $thisrange = $this->range;
+            $range = preg_replace('/\\d+$/', (string) $this->workSheet->getHighestRow(), $thisrange) ?? '';
+            if ($range !== $thisrange) {
+                $this->setRange($range);
+            }
+        }
+
+        return $this;
+    }
+
     /**
      * Get all AutoFilter Columns.
      *
@@ -201,6 +230,7 @@ public function getColumnByOffset($columnOffset)
      */
     public function setColumn($columnObjectOrString)
     {
+        $this->evaluated = false;
         if ((is_string($columnObjectOrString)) && (!empty($columnObjectOrString))) {
             $column = $columnObjectOrString;
         } elseif (is_object($columnObjectOrString) && ($columnObjectOrString instanceof AutoFilter\Column)) {
@@ -230,6 +260,7 @@ public function setColumn($columnObjectOrString)
      */
     public function clearColumn($column)
     {
+        $this->evaluated = false;
         $this->testColumnInRange($column);
 
         if (isset($this->columns[$column])) {
@@ -253,6 +284,7 @@ public function clearColumn($column)
      */
     public function shiftColumn($fromColumn, $toColumn)
     {
+        $this->evaluated = false;
         $fromColumn = strtoupper($fromColumn);
         $toColumn = strtoupper($toColumn);
 
@@ -1002,6 +1034,7 @@ public function showHideRows()
             //    Set show/hide for the row based on the result of the autoFilter result
             $this->workSheet->getRowDimension((int) $row)->setVisible($result);
         }
+        $this->evaluated = true;
 
         return $this;
     }

src/PhpSpreadsheet/Worksheet/AutoFilter/Column.php

@@ -100,6 +100,13 @@ public function __construct($column, ?AutoFilter $parent = null)
         $this->parent = $parent;
     }
 
+    public function setEvaluatedFalse(): void
+    {
+        if ($this->parent !== null) {
+            $this->parent->setEvaluated(false);
+        }
+    }
+
     /**
      * Get AutoFilter column index as string eg: 'A'.
      *
@@ -119,6 +126,7 @@ public function getColumnIndex()
      */
     public function setColumnIndex($column)
     {
+        $this->setEvaluatedFalse();
         // Uppercase coordinate
         $column = strtoupper($column);
         if ($this->parent !== null) {
@@ -147,6 +155,7 @@ public function getParent()
      */
     public function setParent(?AutoFilter $parent = null)
     {
+        $this->setEvaluatedFalse();
         $this->parent = $parent;
 
         return $this;
@@ -171,6 +180,7 @@ public function getFilterType()
      */
     public function setFilterType($filterType)
     {
+        $this->setEvaluatedFalse();
         if (!in_array($filterType, self::$filterTypes)) {
             throw new PhpSpreadsheetException('Invalid filter type for column AutoFilter.');
         }
@@ -202,6 +212,7 @@ public function getJoin()
      */
     public function setJoin($join)
     {
+        $this->setEvaluatedFalse();
         // Lowercase And/Or
         $join = strtolower($join);
         if (!in_array($join, self::$ruleJoins)) {
@@ -222,6 +233,7 @@ public function setJoin($join)
      */
     public function setAttributes($attributes)
     {
+        $this->setEvaluatedFalse();
         $this->attributes = $attributes;
 
         return $this;
@@ -237,6 +249,7 @@ public function setAttributes($attributes)
      */
     public function setAttribute($name, $value)
     {
+        $this->setEvaluatedFalse();
         $this->attributes[$name] = $value;
 
         return $this;
@@ -306,6 +319,7 @@ public function getRule($index)
      */
     public function createRule()
     {
+        $this->setEvaluatedFalse();
         if ($this->filterType === self::AUTOFILTER_FILTERTYPE_CUSTOMFILTER && count($this->ruleset) >= 2) {
             throw new PhpSpreadsheetException('No more than 2 rules are allowed in a Custom Filter');
         }
@@ -321,6 +335,7 @@ public function createRule()
      */
     public function addRule(Column\Rule $rule)
     {
+        $this->setEvaluatedFalse();
         $rule->setParent($this);
         $this->ruleset[] = $rule;
 
@@ -337,6 +352,7 @@ public function addRule(Column\Rule $rule)
      */
     public function deleteRule($index)
     {
+        $this->setEvaluatedFalse();
         if (isset($this->ruleset[$index])) {
             unset($this->ruleset[$index]);
             //    If we've just deleted down to a single rule, then reset And/Or joining to Or
@@ -355,6 +371,7 @@ public function deleteRule($index)
      */
     public function clearRules()
     {
+        $this->setEvaluatedFalse();
         $this->ruleset = [];
         $this->setJoin(self::AUTOFILTER_COLUMN_JOIN_OR);
 

src/PhpSpreadsheet/Worksheet/AutoFilter/Column/Rule.php

@@ -213,6 +213,13 @@ public function __construct(?Column $parent = null)
         $this->parent = $parent;
     }
 
+    private function setEvaluatedFalse(): void
+    {
+        if ($this->parent !== null) {
+            $this->parent->setEvaluatedFalse();
+        }
+    }
+
     /**
      * Get AutoFilter Rule Type.
      *
@@ -232,6 +239,7 @@ public function getRuleType()
      */
     public function setRuleType($ruleType)
     {
+        $this->setEvaluatedFalse();
         if (!in_array($ruleType, self::RULE_TYPES)) {
             throw new PhpSpreadsheetException('Invalid rule type for column AutoFilter Rule.');
         }
@@ -260,6 +268,7 @@ public function getValue()
      */
     public function setValue($value)
     {
+        $this->setEvaluatedFalse();
         if (is_array($value)) {
             $grouping = -1;
             foreach ($value as $key => $v) {
@@ -302,6 +311,7 @@ public function getOperator()
      */
     public function setOperator($operator)
     {
+        $this->setEvaluatedFalse();
         if (empty($operator)) {
             $operator = self::AUTOFILTER_COLUMN_RULE_EQUAL;
         }
@@ -335,6 +345,7 @@ public function getGrouping()
      */
     public function setGrouping($grouping)
     {
+        $this->setEvaluatedFalse();
         if (
             ($grouping !== null) &&
             (!in_array($grouping, self::DATE_TIME_GROUPS)) &&
@@ -359,6 +370,7 @@ public function setGrouping($grouping)
      */
     public function setRule($operator, $value, $grouping = null)
     {
+        $this->setEvaluatedFalse();
         $this->setOperator($operator);
         $this->setValue($value);
         //  Only set grouping if it's been passed in as a user-supplied argument,
@@ -388,6 +400,7 @@ public function getParent()
      */
     public function setParent(?Column $parent = null)
     {
+        $this->setEvaluatedFalse();
         $this->parent = $parent;
 
         return $this;

src/PhpSpreadsheet/Writer/Xlsx/Worksheet.php

@@ -145,7 +145,9 @@ private function writeSheetPr(XMLWriter $objWriter, PhpspreadsheetWorksheet $wor
         $autoFilterRange = $worksheet->getAutoFilter()->getRange();
         if (!empty($autoFilterRange)) {
             $objWriter->writeAttribute('filterMode', 1);
-            $worksheet->getAutoFilter()->showHideRows();
+            if (!$worksheet->getAutoFilter()->getEvaluated()) {
+                $worksheet->getAutoFilter()->showHideRows();
+            }
         }
 
         // tabColor