Report information
The Basics
Id:
22096
Status:
resolved
Worked:
30 minutes
Priority:
Medium/Medium
Queue:

People
Owner:
Nobody in particular
Cc:
AdminCc:

BugTracker
Version Fixed:
(no value)
Version Found:
(no value)
Versions Affected:
(no value)
Versions Planned:
(no value)
Priority:
(no value)
Severity:
S2 Normal
CVSS Score:
(no value)
CVE ID:
(no value)
Component:
(no value)
Area:
feature

Dates
Created:Wed, 15 Sep 2010 14:36:23 -0400
Updated:Wed, 25 Oct 2017 19:00:53 -0400
Closed:Wed, 25 Oct 2017 19:00:53 -0400



This bug tracker is no longer active.

Please go to our Gitlab to submit issues (both feature requests and bug reports) for active projects maintained by Internet Systems Consortium (ISC).

Due to security and confidentiality requirements, full access is limited to the primary maintainers.

Subject: FW: Statistics channel patch
Date: Wed, 15 Sep 2010 14:36:02 -0400
To: <bind-suggest@isc.org>
From: "Timothe Litt" <litt@acm.org>
I was advised that you want suggestions at this address - sorry for any duplication. Timothe Litt ACM Distinguished Engineer --------------------------------------------------------- This communication may not represent the ACM or my employer's views, if any, on the matters discussed. -----Original Message----- From: Timothe Litt [mailto:litt@acm.org] Sent: Monday, September 13, 2010 15:46 To: 'bind-users@lists.isc.org' Subject: Statistics channel patch I have found the statistics channel useful for getting the active zone configuration - this lets my management GUI autoconfigure validity checks and pull-down menus for zones. This will be especially helpful when the dynamic add/delete zone situation is sorted out. But it's useful now because it avoids having the GUI follow all the nested includes of named's config files -- or, as that's so ugly, the separate configuration that I had before. However, I ran into a few issues with this approach: o There is no way to distinguish a built-in zone from a user zone. I currently use serial == 0, but that's not guaranteed. Of course, _bind is in the CH domain, but that still leaves the built-in private address zones (like 8.B.D.0.1.0.0.2.IP6.ARPA/IN/internal). o The vast quantity of statistics are quite a lot for a perl XML parser to handle - especially on older machines. (XML::Simple using XML::Parser on a 400MHz machine takes 20 seconds because the structure of the XML generates zillions of deeply nested anonymous hashes!) o I build quite a few data structures, and want to cache them. But it's really hard (computes and lines of code) to separate out the dynamic data (statistics) from the fairly static (configured zones). In fact, the zone serial number, which is not relevant for configuration, is in that section. So it would require field by field decisions, and on-going maintenance. This resulted in the attached patch, which others may find useful (and I'd be really happy if it made it into the next release in some form). It implements the /config URL on the statistics channel. /config returns everything that's "fairly static". (e.g. If it weren't for dynamic zone addition/deletion, you'd find it in a config file - not a zone file or memory.) It uses exactly the same XML structure as the current / URL and in fact uses the same code. It also adds a <type> tag, with the values "builtin" or "user". (This is still based on serial == 0, but perhaps someone can find a better way to make the determination.) Except for this addition, the / URL's output is unchanged. With this, it's possible to do a simple hash of the returned data and invalidate the cache when it differs from the stored hash of the cached copy. This gets the whole thing down to ~1sec even on my slow machine, which is quite tolerable. It should be easy to add any other useful data, as long as the rule that /config must not have rapidly changing data is obeyed. I did not update the XSL to understand that the dynamic stuff is now optional. It just shows up as blank if you use a web browser. A side note - I was surprised that the <view><zones><zone><name> is of the form "name/class/view", since class and view names are available as separate tags. Easy enough to deal with, but it might be worth considering removing the redundant /class/view from the name at some point. I didn't because I wanted to stay compatible with the mainline. The patch only looks large - as kdiff3 would show, it's mostly indentation where I skip over the unwanted statistics when responding to the new URL. About 16 non-whitespace changes, mostly 1 line. Enjoy. --------------------------------------------------------- This communication may not represent the ACM or my employer's views, if any, on the matters discussed.

Message body not shown because it is not plain text.

Thank you for your feedback on the XML channel.

From a design point of view, I intentionally did not split the data apart but instead returned the whole thing.  This was intentional until we could make it into a more RESTful style.

It is probably time to actually take on the work of splititng the data into chunks.  A simple RESTful API for gathering information about a server, in my mind anyway, was along the lines of:

/views -- returns a list of views
/views/_default -- returns information about a specific view
/views/_default/zones  -- returns a list of zones in a particular view
/views/_default//zones/isc.org -- returns detailed information about this particular zone

and so on.  I am not certain if the /config, /stats, or /status information should be before or after the URIs in this simple example.  My gut tells me after.

--Michael


Subject: RE: [ISC-Bugs #22096] FW: Statistics channel patch
Date: Wed, 15 Sep 2010 20:17:28 -0400
To: <bind-suggest@isc.org>
From: "Timothe Litt" <litt@acm.org>
Michael, Thanks for the reply. I'm not too fussy about exactly what things look like, as long as it can function. Here are my 3 cents: I think config, stats, status are orthogonal selectors to the data's structure - so why not put them in the query string? When one knows what one needs, it's best not to have to make multiple requests. (.g. /views to get the view list, then /views/eachview/zones to get the zone list, then /views/eachview/zones/zone/eachzone to get the zone type...) For my present use, I build a dropdown for actively served, user (non-default) zones. This happens fairly frequently. So it's important to keep it efficient. I don't want to make more than one query to the server to get this configuration data. So maybe something like: /views/*/zones?view.zone.type=user&DataClass=config would work. The idea is that the query could be any xml path=desired value, and DataClass is a metavariable. Alternatively, if you want to think about the Data class as a part of the structure, I'd put in front, not after. This because the rest of the path is going from the most general to the most specific. And /config/views/*/zones/* ... means "just config data at every level", not just at the last one... Of course, my litmus test is whether I can issue one request and get what I need for my application. Perhaps an example to test your design against would help. The outline of my code is: Fetch config checksum if matches cache, use cache (don't even parse the string) else update cache & use result. Stripping out error handling and a bunch of other details (for clarity), my current code looks like: my $config = LWP::Simple::get( "http://$masterNs:8053/config" ); my $checksum = Digest::SHA1::sha1($config); my $cfgname = dirname(Cwd::realpath( $0 )) . '/DnsConfig.db'; My( $cfg, %zones, %validzone ); eval { cfg = Storable::lock_retrieve($cfgname); }; if( $cfg && $cfg->[0] == 1 && $cfg->[1] eq $checksum ) { %zones = @{$cfg->[2]}; %validzone = @{$cfg->[3]}; ... } else { my $xs = XML::Simple->new(); # Extract views (because it speeds up parsing) $config =~ m/(<views>.*?<\/views>)/ms or die "Can't find views\n";; my $views = $1; $views = $xs->parse_string( $views ); # Build %zones from server's active view/zone list for my $view (keys %{$views->{view}}) { my $zones = $views->{view}{$view}{zones}{zone}; my( @fwd, @rev ); for my $z (keys %$zones) { next if( $zones->{$z}{type} ne 'user' ); next if( lc( $zones->{$z}{rdataclass} ) ne 'in' ); next unless( $z =~ m!^(.*)/IN/$view$! ); $z = lc( $1 ); if( $z =~ /^(.*)\.in-addr.arpa$/ ) { my @a = reverse( split( m/\./, $1 ) ); my $a = join( '.', @a ); # Split out [0] private and [1] public IP zones # note that is_privateip returns a classification code; ! reduces to 0 or 1 push @{$rev[!is_privateip($a . scalar('.0' x (4-@a) ))]}, [ $a, "$z/$view" ]; $validzone{"$z./$view"} = $a; } else { # Forward zones push @fwd, [ $z, "$z/$view" ]; $validzone{"$z./$view"} = $z; } next unless( @fwd || @rev ); # Skip empty views $zones{$view}{forward} = [ @fwd ] if( @fwd ); $zones{$view}{reverse}{private} = sortip @{$rev[0]} if( $rev[0] ); $zones{$view}{reverse}{public} = sortip @{$rev[1]} if( $rev[1] ); } } Storable::lock_store( [ 1, # [0] Version $checksum, # [1] Checksum of received network data [%zones], # [2] [%validzone], # [3] ... ], $cfgname ); } --------------------------------------------------------- This communication may not represent my employer's views, if any, on the matters discussed. -----Original Message----- From: Michael Graff via RT [mailto:bind-suggest@isc.org] Sent: Wednesday, September 15, 2010 16:03 To: litt@acm.org Subject: [ISC-Bugs #22096] FW: Statistics channel patch Thank you for your feedback on the XML channel. >From a design point of view, I intentionally did not split the data apart but instead returned the whole thing. This was intentional until we could make it into a more RESTful style. It is probably time to actually take on the work of splititng the data into chunks. A simple RESTful API for gathering information about a server, in my mind anyway, was along the lines of: /views -- returns a list of views /views/_default -- returns information about a specific view /views/_default/zones -- returns a list of zones in a particular view /views/_default//zones/isc.org -- returns detailed information about this particular zone and so on. I am not certain if the /config, /stats, or /status information should be before or after the URIs in this simple example. My gut tells me after. --Michael
Subject: RE: [ISC-Bugs #22096] FW: Statistics channel patch
Date: Thu, 16 Sep 2010 05:59:05 -0400
To: <bind-suggest@isc.org>
From: "Timothe Litt" <litt@acm.org>
One more thought - another thing that might be useful would be a way to register for event notifications; specifically configuration changes. If that were possible, I could have a monitoring process that registered for config changes (e.g. zones added/removed, or status changed, or...), and only fetched the XML when it started or was notified. It would then update the GUI data - or whatever. The caller would do the usual html stuff to keep the connection open. That might look like: /views/*/zones?view.zone.type=user&DataClass=config&RegisterForUpdates=1 In this case, named would send the xml, but keep the query on a trigger list. When the config was updated (or if status was selected and status chaged), it would regenerate the XML and send it again (server push style). For statistics, I think there would be less benefit - one could have a RefreshInterval - but the client can keep a timer just as easily and simply ask again. Something like this would reduce the load on everyone - named isn't serving redundant queries, and its load scales with the number of applications (not the number of application instances), the monitor only wakes when something happens, and the GUI (or other data collector) simply reads the cached, pre-processed data. This is more than I currently require, but might well be better for the wider community. FWIW, Timothe Litt ACM Distinguished Engineer --------------------------------------------------------- This communication may not represent the ACM or my employer's views, if any, on the matters discussed. -----Original Message----- From: Michael Graff via RT [mailto:bind-suggest@isc.org] Sent: Wednesday, September 15, 2010 16:03 To: litt@acm.org Subject: [ISC-Bugs #22096] FW: Statistics channel patch Thank you for your feedback on the XML channel. >From a design point of view, I intentionally did not split the data >apart but instead returned the whole thing. This was intentional until we could make it into a more RESTful style. It is probably time to actually take on the work of splititng the data into chunks. A simple RESTful API for gathering information about a server, in my mind anyway, was along the lines of: /views -- returns a list of views /views/_default -- returns information about a specific view /views/_default/zones -- returns a list of zones in a particular view /views/_default//zones/isc.org -- returns detailed information about this particular zone and so on. I am not certain if the /config, /stats, or /status information should be before or after the URIs in this simple example. My gut tells me after. --Michael
Subject: RE: [ISC-Bugs #22096] FW: Statistics channel patch
Date: Mon, 1 Nov 2010 16:08:07 -0400
To: <bind-suggest@isc.org>
From: "Timothe Litt" <litt@acm.org>
I haven't updated my patch - but have realized that one other piece of configuration information that will be needed is the zone type - as in {master, slave, stub}. In my immediate case, this is because a GUI shouldn't offer the opportunity to modify a stub, and a view containing only stub and perhaps built-in zones isn't interesting to (most) humans. I'd appreciate it if this was included in your work. Thanks. --------------------------------------------------------- This communication may not represent my employer's views, if any, on the matters discussed.