Skip to content

Enable treating IP ranges as fully populated #9763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nem1989 opened this issue Jul 18, 2022 · 21 comments
Closed

Enable treating IP ranges as fully populated #9763

nem1989 opened this issue Jul 18, 2022 · 21 comments
Assignees
Labels
complexity: medium Requires a substantial but not unusual amount of effort to implement netbox status: accepted This issue has been accepted for implementation type: feature Introduction of new functionality to the application
Milestone

Comments

@nem1989
Copy link

nem1989 commented Jul 18, 2022

NetBox version

v3.2.6

Feature type

New functionality

Proposed functionality

Improve IP ranges functionality by implementing these features:

  • An option to display particular IP ranges in IP address lists.
  • An option to mark an IP range as restricted/reserved/fully utilized. IP addresses within such ranges will not show up as available in any IP lists.

Use case

This would be extremely useful for DHCP ranges for instance. Right now nothing stops netbox users from using "free" IP addresses reserved for particular IP ranges.

Example: I have an IP range defined which describes DHCP range in one of my subnets, but when browsing IP address lists nothing indicates that these addresses are reserved, thus they can be assigned to any device or VM leading to an IP conflict.

There is a workaround: bulk create IP addresses for the whole range and set a corresponding role for them all. But in this scenario users have to manually match ranges and IP addresses in case of range changes and accidents may happen due to human factor. Also a lot of unnecessary information is stored in the database and displayed in IP lists especially for large IPv6 prefixes where there can be thousands of reserved addresses in one prefix.

With my proposal implemented one could mark an IP range as reserved and users would not be allowed to allocate addresses from reserved ranges anymore or atleast will be notified that these addresses are reserved. It would also decrease amount of excessive information in IP lists, dramatically in some cases.

These features are optional and will not break existing databases.

Database changes

Some new boolean fields for IP range description will be needed to implement this.

External dependencies

No response

@nem1989 nem1989 added the type: feature Introduction of new functionality to the application label Jul 18, 2022
@nem1989 nem1989 changed the title Improve IP ranges functionality Improve IP ranges functionality (reserved IP ranges) Jul 20, 2022
@nem1989 nem1989 changed the title Improve IP ranges functionality (reserved IP ranges) Implement reserved IP ranges Jul 20, 2022
@jeremystretch
Copy link
Member

An option to display particular IP ranges in IP address lists.

Can you elaborate on this? How do you anticipate this working? What would the UI look like with this change in place?

An option to mark an IP range as restricted/reserved/fully utilized

This has already been captured in #7947.

@jeremystretch jeremystretch added the status: revisions needed This issue requires additional information to be actionable label Jul 27, 2022
@nem1989
Copy link
Author

nem1989 commented Aug 3, 2022

An option to display particular IP ranges in IP address lists.

Can you elaborate on this? How do you anticipate this working? What would the UI look like with this change in place?

An option to mark an IP range as restricted/reserved/fully utilized

This has already been captured in #7947.

I'm no UI designer but from my viewpoint in IP lists (doesn't matter if it is in prefix, filter or just all IPs list) there could be placeholders for IP ranges just like there are now for available ranges.

Like this:
10.177.100.1
10.177.100.2
100 IPs available (green)
100 IPs reserved (with reserved IP range Role in Role column) (yellow/red/configurable for each IP range?)
10.177.100.203
...

Wether to show IP range in lists or not could be configured with either a checkbox or dropdown list inside IP range edit menu. With dropdown this functionality can be extended with reasons why range is reserved/utilized (if it is just utilized or intended to be used for a special purpose).

If there is an IP address within range it should be showed too.

Like this:
10.177.100.1
10.177.100.2
100 IPs available
49 IPs reserved
10.177.100.151
50 IPs reserved
10.177.100.203

Reserved ranges should be treated like available on-click - user can assign an IP from reserved range by clicking on it's placeholder in the list. But there should be some kind of a prompt when new IP is on reserved range.
Like:
"This IP is reserved, are you sure?" or a red/yellow/contrast informational note in IP edit menu saying that this IP is reserved so that reserved IPs could be used only on purpose and not accidentally.

There also could be a checkbox in IP range edit menu toggling if IP range is displayed in IP lists or not. Defaults to not so that nothing is changed for users not needing this functionality.

@jeremystretch jeremystretch removed the status: revisions needed This issue requires additional information to be actionable label Aug 8, 2022
@CharlesSerrett
Copy link

This may increase the scope of the issue but I'd like to be able to assign DHCP and SLACC to IP ranges and then see this in the IP addresses view.
image

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions github-actions bot added the pending closure Requires immediate attention to avoid being closed for inactivity label Nov 13, 2022
@elipsion
Copy link

I'm unsure about the maintainers' opinion about this feature, since the tag removal on Aug. 8. The flowchart on feature intake seems to have lost track on "In scope for core?"

In any case, we also see a huge potential in this. Right now we have quite vast (/16 and larger) networks containing different DHCP scopes together with blocks of static assignments. It's a bit unwieldy to bulk create 10k addresses with status DHCP to have the usage calculator work and make sure nobody accidentally places a static IP in the DHCP scope.

@fercalbla
Copy link

I'm unsure about the maintainers' opinion about this feature, since the tag removal on Aug. 8. The flowchart on feature intake seems to have lost track on "In scope for core?"

In any case, we also see a huge potential in this. Right now we have quite vast (/16 and larger) networks containing different DHCP scopes together with blocks of static assignments. It's a bit unwieldy to bulk create 10k addresses with status DHCP to have the usage calculator work and make sure nobody accidentally places a static IP in the DHCP scope.

Same opinion here, we have lot of ranges for DHCP and this will be a very good feature to avoid people assigning these IPs

@jeremystretch jeremystretch added status: under review Further discussion is needed to determine this issue's scope and/or implementation and removed pending closure Requires immediate attention to avoid being closed for inactivity labels Dec 15, 2022
@dutchman80
Copy link

This may increase the scope of the issue but I'd like to be able to assign DHCP and SLACC to IP ranges and then see this in the IP addresses view. image

Or at least not show as "Available" in the IP Addresses tab of the Prefix view, like it does now

@iamjla
Copy link

iamjla commented Feb 15, 2023

@jsenecal closed the above mentioned issue with the notice to continue here.

We've also got some proposals to shape this feature:

We would suggest "IP Ranges" gets the following 2 new booleans to add this functionality:

  • "Exclude from Available IPs"
    This still allows creation of IPs of this IP Range but prohibits them from being used in available ip logic (API&UI)
  • "Block creation of IPs in this Range"
    As the name suggests, any IP creation in this range should be blocked

tl;dr of the use case is saving on database entrys, as a lot comments in this issue mention. For a full explanation from our standing i'd suggest taking a look at #11678
The comment of @do9xe in #11678 adds some use cases as well

@do9xe
Copy link

do9xe commented Mar 2, 2023

I was just looking at the code and found a way how this might be possible.
There is a function called add_available_ipaddresses() which creates a list of tuples that represent all the blocks that are free. You'd need to add ip-ranges into that function and add an additional element to the touples to distinguish between "free" and "reserved", maybe even a fourth so you could display the name of the range there.

I'd like to look into this and propose a pull-request, if this feature request is accepted.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 1, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

@github-actions github-actions bot added the pending closure Requires immediate attention to avoid being closed for inactivity label Jun 1, 2023
@do9xe
Copy link

do9xe commented Jun 2, 2023

According to the contribution guide I'm supposed to wait.

You're welcome to volunteer to implement your FR, but don't submit a pull request until it has been approved.

Now this FR/Issue is marked as pending closure again. I'm not quite sure if anyone from the maintainers has an eye on this.

@nem1989
Copy link
Author

nem1989 commented Jun 6, 2023

I see I should not be "bumping this" but I believe something went wrong here. Can this issue be under review and pending closure at the same time? It was not reviewed by maintainers and this will just lead to a creation of a new duplicate feature request after this one is closed.

@DanSheps DanSheps added needs milestone Awaiting prioritization for inclusion with a future NetBox release and removed status: under review Further discussion is needed to determine this issue's scope and/or implementation pending closure Requires immediate attention to avoid being closed for inactivity labels Jun 6, 2023
@DanSheps
Copy link
Member

DanSheps commented Jun 6, 2023

Going to mark this as needs milestone as there clearly is a decent amount of interest in this.

@nem1989 Thank you for trying to follow the rules, however a thing to keep in mind is that pending closure is automatically added when there hasn't been much activity on an issue in a certain amount of time.

@netbox-community netbox-community deleted a comment from peter2c Oct 13, 2023
@DanSheps DanSheps added status: under review Further discussion is needed to determine this issue's scope and/or implementation and removed needs milestone Awaiting prioritization for inclusion with a future NetBox release labels Oct 13, 2023
@parentsb
Copy link

Is there any updates on this? I'm keeping this page in bookmarks and checking it every month.

@jmiezitis
Copy link

We have mixed usage prefixes where one part of the prefix is allocated to openstack which deploys from that range using DHCP while other parts of the range are manually configured for different infrastructure. Doing what CharlesSerret suggests would be best from my and my teams point of view.
At the moment, before allocating an address from the IP Address tab using the IP's Available button, we have to check the Child Prefix tab and Child Range tab to see if the IP is part of a Child Prefix pool or a Child Range. To help avoid mistakes this needs to be reduced to just viewing the IP Addresses tab where we should be able to see, in one place, how addresses are currently allocated;

  • free and not allocated to a child range or child prefix pool with a button to click to reserve/use and address
  • free and allocated to a child range or child prefix pool with a button to click to reserve/use and address
  • reserved/used and not allocated to a child range or child prefix pool
  • reserved/used and allocated to a child range or child prefix pool

@DanSheps
Copy link
Member

To summarize this FR to be clear about the intended changes, this FR will:

  • Add a boolean to disable the allocation of IP addresses, so long as those addresses fall within the range's serviced prefix and VRF
  • Add UI indicators for this range to replace the "available IP(s)" with "reserved IP(s)" where IPs are impacted by these reseved ranges

Proposed changes:

  • Add new boolean on the model to denote that a range is not available for allocation in netbox
  • Add form fields for the specified boolean
  • Modify the detail view of a prefix to retrieve ranges within the prefix and change any "available IP" indicators to "reserved"
  • Modify the form clean() to disallow allocation of addresses within a reserved range

This sound somewhat reasonable to everyone?

@nem1989
Copy link
Author

nem1989 commented Oct 25, 2023

To summarize this FR to be clear about the intended changes, this FR will:

  • Add a boolean to disable the allocation of IP addresses, so long as those addresses fall within the range's serviced prefix and VRF
  • Add UI indicators for this range to replace the "available IP(s)" with "reserved IP(s)" where IPs are impacted by these reseved ranges

Proposed changes:

  • Add new boolean on the model to denote that a range is not available for allocation in netbox
  • Add form fields for the specified boolean
  • Modify the detail view of a prefix to retrieve ranges within the prefix and change any "available IP" indicators to "reserved"
  • Modify the form clean() to disallow allocation of addresses within a reserved range

This sound somewhat reasonable to everyone?

Sounds great!

There should be a link to a range itself from IP list views. Clicking available IP creates a new IP and clicking reserved IP would open a range view.

It also might be nice to have an IP range description/role indicator of some sort in IP list views so that one could see WHY is it reserved.

For example:
10 IPs
10 IPs DHCP
10 IPs Private
...

@elipsion
Copy link

  • Modify the detail view of a prefix to retrieve ranges within the prefix and change any "available IP" indicators to "reserved"

I think it would be nice if the indicator displayed the status from the range object, instead of just showing a generic "reserved" keyword.

@netbox-community netbox-community deleted a comment from maudechai Mar 7, 2024
@netbox-community netbox-community deleted a comment from parentsb Mar 7, 2024
@netbox-community netbox-community deleted a comment from M4rlus Mar 7, 2024
@netbox-community netbox-community deleted a comment from savaloi Mar 7, 2024
@netbox-community netbox-community deleted a comment from jronnblom Mar 7, 2024
@jeremystretch jeremystretch added needs milestone Awaiting prioritization for inclusion with a future NetBox release status: backlog Awaiting selection for work and removed status: under review Further discussion is needed to determine this issue's scope and/or implementation labels May 14, 2024
@jeremystretch jeremystretch added the complexity: medium Requires a substantial but not unusual amount of effort to implement label May 23, 2024
@jeremystretch jeremystretch added the netbox label Nov 1, 2024 — with Linear
@jeremystretch jeremystretch removed the needs milestone Awaiting prioritization for inclusion with a future NetBox release label Feb 20, 2025
@jeremystretch jeremystretch added this to the v4.3 milestone Feb 20, 2025
@jeremystretch jeremystretch self-assigned this Apr 1, 2025
@jeremystretch jeremystretch added status: accepted This issue has been accepted for implementation and removed status: backlog Awaiting selection for work labels Apr 1, 2025
@netbox-community netbox-community deleted a comment from Mefevre Apr 1, 2025
@jeremystretch
Copy link
Member

I'm currently working on this. I've introduced a new boolean named mark_populated on the IPRange model. I've opted for this name in part to avoid confusion with the "reserved" status, which is meant to indicate planned future use.

If true, mark_populated will cause NetBox to treat the range as though every IP address within the range already exists (even though the individual database records for those IPs do not). Note that this is distinct from the purpose of the existing mark_utilized boolean field, which instructs NetBox on how to represent a range when calculating the utilization of a parent object.

The child IP addresses view for a prefix has been extended to show ranges, IP, and available space in the table (see below). Only IP ranges with mark_populated=True will be included in the list.

Image

This leaves us with four possible flag combinations for an IP range:

mark_populated mark_utilized Can create IPs in range? Utilization reported
- - Yes Calculated based on number of IPs
- No 0% (no IPs exist)
- Yes 100%
No 100%

The first case applies when an IP range is used merely for organization of the space within a prefix.

The second case is admittedly a bit esoteric, but I can see it being useful to represent e.g. recently reclaimed IP space.

The third case applies when you want to manage IPs within a range but always consider the range itself as utilized for planning purposes (probably the most common use case).

The final case is for effectively deferring the management of an IP range from NetBox to some other system (e.g. a DHCP server).

@DanSheps
Copy link
Member

DanSheps commented Apr 3, 2025

Thanks for the summary table and the breakdown of what each case should be used for. Based on the description, I think this will accomplish what everyone is looking for.

@jeremystretch jeremystretch changed the title Implement reserved IP ranges Enable treating IP ranges as fully populated Apr 3, 2025
@pheus
Copy link
Contributor

pheus commented Apr 7, 2025

First of all — huge thanks for working on this! This is one of the most requested features from my colleagues, and I'm really happy to see it coming to life.

Also, I really appreciate the summary table and detailed explanation — it's super helpful for understanding the intent behind the design.

That said, I find the distinction between mark_populated and mark_utilized a bit confusing at first glance. In particular, the second case (mark_populated=True but mark_utilized=False) feels — functionally — like what I'd expect from a "reserved" range: addresses that don’t physically exist in the DB, can’t be assigned, but also don’t affect utilization.

I wonder if we might improve intuitiveness by considering a refactor to a ChoiceField. For example, we could have explicit states like:

Value UI Display Description Can create IPs? Utilization Reported
None Default behavior Yes Calculated
populated Fully populated & utilized Treat range as if every IP exists (no additions allowed) No 100%
utilized Utilized only Affects utilization calculation without restricting new IPs Yes 100%
reserved Reserved Blocks IP creation without marking as utilized No 0%

This approach might make it easier for users to understand the different states at a glance without having to mentally map boolean combinations.

Of course, this is just an idea! I can absolutely see why you might prefer booleans for flexibility or backwards compatibility reasons, but I figured I'd share this thought in case it sparks something useful.

Thanks again for all the work here — I'm really looking forward to this feature!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
complexity: medium Requires a substantial but not unusual amount of effort to implement netbox status: accepted This issue has been accepted for implementation type: feature Introduction of new functionality to the application
Projects
None yet
Development

No branches or pull requests