Structure priority option in structure overlapping region

[weiliangjin2021]

In many RF smulations, metallic structures are usually very thin: thickness much smaller than grid size. We can still accurately simulate them by applying PEC conformal meshing and SIBC techniques. However, since we slightly expand the structures in the solver (not really now, but a similar strategy), when the metallic structures are not added later compared to the touching dielectric structures in the structure list, those metallic structures will be completely missing in Yee-grid representation. It is a very common issues when people setup an RF simulation, even among our internal developers.

As discussed here, the scope of this PR:

• Add a priority field to AbstractStructure. The default value is None.
• Add a field structure_priority_mode to Scene. It sets priority of structures that have priority=None. Right now, there are two modes:
  • equal that sets priority=0;
  • conductor that sets priority=90 for lossyMetaMedium, and 100 for PEC, and 0 for others
• A cached_property sorted_structures in Scene to be applied in places where sim.structures was used
• MeshOverideStructures will have default priority=0, regardless of the value of structure_priority_mode.
• In meshing, structures = sorted_structure + sorted_mesh_override_structures
• Priority of 2d medium
• Add tests

[yaugenst-flex]

Haven't had time to review properly yet but a quick thought: Maybe it would make sense if we introduced a numbers-based priority to structures in general? By default they'd all have the same, and then priority is decided on ordering, but giving a larger number would mean higher priority. This is a pretty common approach I think, for example matplotlib does it for zorder. When dealing with metals, these could get a higher priority by default for example? This seems like it would be a useful feature to have in general. Hopefully it would also obviate the need for a sim.sorted_structures?

[weiliangjin2021]

Haven't had time to review properly yet but a quick thought: Maybe it would make sense if we introduced a numbers-based priority to structures in general? By default they'd all have the same, and then priority is decided on ordering, but giving a larger number would mean higher priority. This is a pretty common approach I think, for example matplotlib does it for zorder. When dealing with metals, these could get a higher priority by default for example? This seems like it would be a useful feature to have in general. Hopefully it would also obviate the need for a sim.sorted_structures?

That's actually my initial plan. But the concern is that this feature will barely be used, and people will only use the automated priority approach. @tomflexcompute also brought up this when he is using a certain photonic simulation software.

[weiliangjin2021]

When dealing with metals, these could get a higher priority by default for example? This seems like it would be a useful feature to have in general. Hopefully it would also obviate the need for a sim.sorted_structures

For backward-compatibility, we cannot by default assign higher priority to metal. So we will still need an additional field like the one added in this PR that sets the automation behavior: by structure list order or material property. To that regard, I think we can just have this additional field for automation for now, and adds the manual priority number when requested in the future.

Now regardless of the approach, I think we'll still need .sorted_structures internally. Otherwise, there will be even more changes to the code: significant changes in plotting, mesher, solver processing.

[momchil-flex]

Yeah I'm not sure honestly, I see the advantage of the current approach in that it's easier for the user to set, but it does feel like we'll just keep running into the need to have a priority setting in the structures. It seems like it could already make at least the internal code a bit better, and then if we have it internally, we might expose it anyway.

If we were to do that, how might the metallic option combine with it?

[momchil-flex]

By metallic we explicitly mean PEC or SIBC but not e.g. usual dispersive metals?

[weiliangjin2021]

we don't have any special subpixel methods applied to usual dispersive metals. So we only mean "good conductor" by "metallic" here.

[weiliangjin2021]

Now it's renamed to conductor

[momchil-flex]

There was some rearrangement done for 2D media too, right? I think it was to put the equivalent volumetric structures on top. I wonder how this plays with these changes, and if it's at all possible to unify these rearrangements more - also to clarify even better to the user what exactly happens in various cases (metals + 2D media? very strange case I guess but still)

[weiliangjin2021]

but it does feel like we'll just keep running into the need to have a priority setting in the structures.

Could you give more contexts on what other occasions that requires a manual priority setting?

[weiliangjin2021]

it seems like it could already make at least the internal code a bit better, and then if we have it internally, we might expose it anyway.

What's your thought on improving the internal codes with manual priority setting? I think we'll still need the sorted_structures to be used anywhere that requires the structure lists so that we don't sort it again in those places?

If we were to do that, how might the metallic option combine with it?

Yeah, that's my question too. In this approach, I think the priority field in structure will be Union[float, "auto"], and we set the default as auto. Then we'll still need an additional field in simulation class to indicate the behavior of auto.

[yaugenst-flex]

I think the priority should just be an int, with some number, e.g. 0, by default for everything. This would be identical to the current behavior and backward-compatible. I guess one use case is simply ergonomics. Sometimes it's nice to not have to worry about the order in which you put your structures into a list. Another one that I can think of is when we are inserting new structures into simulations. In inverse design, we often create a base simulation into which we then insert structures. You might want to insert a structure that is not at the very end of the list but insert it such that it covers some part of the base simulation but gets clipped by another part. That's something that would probably be easier to do with these priorities.

[weiliangjin2021]

@momchil-flex One question on meshing: in this part, we have both physical structures and mesh override structures. There are two directions for us to handle this:

• We don't distinguish between the two structures, and sort them all together;
• We sort physical structures and mesh override structures separately, and structure list will be sorted physical structures + sorted override structures. So the override structures take higher precedence.

Personally, I prefer the 2nd approach. For the 1st approach, since for now the default priority of override structures is 0, and metallic physical structures have higher priority, override structures might never take effect.

[weiliangjin2021]

Another question to @dmarek-flex and @caseyflex: in the conductor mode, I plan to set priority=90 for lossyMetaMedium, 100 for PEC, and 0 for others. For Medium2D, and lumped elements that generate Medium2D, does it make more sense to set highest priority for Medium2D, like 110? In that case, those lumped elements won't be overridden by the bulk metal parts.

[momchil-flex]

@momchil-flex One question on meshing: in this part, we have both physical structures and mesh override structures. There are two directions for us to handle this:

• We don't distinguish between the two structures, and sort them all together;
• We sort physical structures and mesh override structures separately, and structure list will be sorted physical structures + sorted override structures. So the override structures take higher precedence.

Personally, I prefer the 2nd approach. For the 1st approach, since for now the default priority of override structures is 0, and metallic physical structures have higher priority, override structures might never take effect.

2nd approach makes sense to me too. I think for some very esoteric situations where someone wants to have a regular structure above a meshing override one, they can just also add the regular structures in the override structures list.

[caseyflex]

Another question to @dmarek-flex and @caseyflex: in the conductor mode, I plan to set priority=90 for lossyMetaMedium, 100 for PEC, and 0 for others. For Medium2D, and lumped elements that generate Medium2D, does it make more sense to set highest priority for Medium2D, like 110? In that case, those lumped elements won't be overridden by the bulk metal parts.

It looks to me like currently, when a Medium2D is encountered, it stays where it is in the list of structures. This choice was made to allow defining structures like in this notebook:

https://docs.flexcompute.com/projects/tidy3d/en/latest/notebooks/Bandstructure.html

where you place a uniform medium (which could be a Medium2D), and then put air holes into it

[dmarek-flex]

Another question to @dmarek-flex and @caseyflex: in the conductor mode, I plan to set priority=90 for lossyMetaMedium, 100 for PEC, and 0 for others. For Medium2D, and lumped elements that generate Medium2D, does it make more sense to set highest priority for Medium2D, like 110? In that case, those lumped elements won't be overridden by the bulk metal parts.

Good question, current behavior of lumped elements is that they are always appended to the all_structures list during volumetric_structures, so none of your changes should change that. I also think that it is ideal to prioritize lumped elements in this way.

For general Medium2D, I think they should be prioritized in the same manner as regular Medium are in this PR. So giving priority to PEC/LossyMetal, if desired by the user. But otherwise they maintain their order based on when they are added to the structures list.

[weiliangjin2021]

Good question, current behavior of lumped elements is that they are always appended to the all_structures list during volumetric_structures, so none of your changes should change that. I also think that it is ideal to prioritize lumped elements in this way.

In conductor mode, the position of lumped elements in all_structures list doesn't guarantee higher priority, because internally we will sort the structure list based on the priority. If we don't assign higher priority to them, they can be overridden by PEC etc.. So perhaps I'll assign them the same priority as PEC? When structures are of the same priority, the latter ones take precedence.

For general Medium2D, I think they should be prioritized in the same manner as regular Medium are in this PR. So giving priority to PEC/LossyMetal, if desired by the user. But otherwise they maintain their order based on when they are added to the structures list.

OK, we will not assign them special priority. Their priority will only be boosted, if they are 2D PEC.