Description
In this clip from our Networking Field Day 30 presentation, Nitin Kumar, Selector Co-founder and CTO, explains how we connect to metadata stores.
Video length: 2:58
Speakers: Jordan Martin, Principal Solutions Architect, WWT; Pete Robertson, Principal Network Consultant, AHEAD; Nitin Kumar, Co-founder and CTO, Selector
Transcript
[Video text]
Networking Field Day 30 Selector AI
A Deep Dive into Selector AI
Nitin Kumar
January 20, 2023
Jordan Martin: One of the things that I think also needs to be considered is there are a lot of static elements that contribute to the analysis of our network—things that don’t change, things that don’t show up in logs, things that don’t show up as metrics—that are kind of steady state but have an impact on the analysis of what’s going on. [agreement interjected] How do you correlate not only disparate sets of data, because that’s an incredibly challenging problem like, to your point, numbers … sentences. I’m sure there are probably 15 other ingestion formats that could exist beyond those two. But then how do you take into consideration the things that are static that are not generating those things? [agreement interjected]
Pete Robertson: There’s a context to what is taking place right, and the context is not necessarily ingested through telemetry or SNMP. To Jordan’s point, how are you making the correlation in light of that context to drive the actual wins?
Nitin Kumar: Yes. Javier is sitting at the back, and he pointed this out just before the presentation yesterday. You need to include metadata as well in this light. The metadata is a very important piece of information that generally gets missed in our system. Since I don’t have a slide description for that, we have a first-class integration to metadata stores. NetBox is the most commonly used store these days, and we connect to NetBox. We use the NetBox APs, and suck in that information. Those tables exist in the system as a data element. And as those streams are coming in, they start getting joined and used as well.
Jordan Martin: How do you reconcile that with your strategy of ingesting all the data only as long as you need it to build insights? Are you just redoing that process on some interval? How does that work?
Nitin Kumar: Two parts of that. The meta stores are not voluminous.
Jordan Martin: Okay. You treat that data a little bit differently.
Nitin Kumar: Yeah. Because even though we don’t expose it to the users—these retention policies and how long something is stored—our admins do control that. We have some amount of control where these meta stores can live forever. That’s number one because they’re a drop in the ocean when it comes to comparing the other stuff.
The other part is, even in these meta stores, we have a period set to a day sometimes … very infrequent. And the third option is where some of us have a customer who says, “Hey, I know when I’ve updated my meta store. Just give me a button that I know I’ve updated it here, and I’m going to make an API call that will call your ingest on demand, or I’m going to log into the portal and do things. A combination of these three things allows us to get metadata into the platform.