HOME | COUNTRIES | CITIES | COMPANIES | SPORTS | CARS
news videos images websites wiki



Good afternoon welcome glad to see here i am steve Bartos I manage the predictive. Analytics team the small but awesome predictive analytics team at Worthington Industries here with my colleague will Davis a member of said. Team and we're going to talk to you really we're going to talk about two things first make a couple promises when we're gonna. Absolutely help you come down off the high of last night now I know there's a pretty sick party but we're gonna we made a guarantee to you I just talked to my tableau rep if we can't recreate the excitement and entertainment value that you had last night your conference is. On us and actually on us I mean on Ron so Ron couldn't be here I figured he'd he'd help me keep that promise so we'll make that promise to you the second thing we're going to talk about is how we and really by we I mean will has have supercharged tableau with our to help our quoting. Group get much better at how we price in the steel processing division and here's what we're going to talk about first I'll lead you on a whirlwind tour brief history of analytics at Worthington keeping in mind just to help you get. An idea of how we got to where we are with this specific project and then I'll pass the mic over to to will and he'll talk about some of the early modeling attempts he calls and misfires but some of the learnings around how we were going to model against this problem how he built an interactive. User interface what you're saying well of course isn't that what tableau provides but it allows the user to select parameters of interest. While they're quoting and then generates the model based on those parameters and also gives the user a view of the health of the model the. Robustness of the model maybe you don't have enough data in the model so you're gonna see that. Red light that says hey you might want to just go to something more descriptive but some really cool stuff and then at the end we'll just geek out this would be like the Trombone Shorty part of the show just talking about the stack and some of the IT. Work that Will has done with our team to put together the technology to make this happen all right the brief history of analytics so will and I and another woman we perform the predictive analytics team in the steel processing division a. Month ago we were elevated I guess it's elevated to a corporate function but for the sake of this this discussion we're talking about the steel processing division at Worthington Industries I see some of your faces you're thinking steel processing sounds fascinating it absolutely. Is what we do is we buy steel we're the fourth largest purchaser of steel outside the Detroit Three the. Automakers we buy steel and then we do stuff to steel we process it right we heat it up to make it more for mobile we clean it we cut it we coat it do all sorts of cool stuff and then that goes on to a variety of end uses there's uses in construction. Heavy machinery grain bin culvert but for the birth of analytics that whirlings and what's important to focus on is the fifty to sixty percent of our business that ends up in an automobile something is utilitarian is your seat belt buckle something more maybe engineered like the rail that your your seat rides on or. Then like the super engineered products that end up in your transmission for example you go back ten years ago this is an actual picture I know tableau doesn't like it when use. Clip art this is an actual picture of the first predictive analyst at Worthington Industries his name was son pre so about ten years ago we hire son pre and you can see he saw tremendous. Opportunity for changing how we do what we do in steel processing by using better using data and analytics but back to the the the automotive finished goods that we produce there was strategically that part of. The business we were struggling in fifty to sixty percent of our business we do it we do a very poor inconsistent job forecasting hope and so there. Was strategic alignment on how do we better improve our forecasting for our automotive finished goods right and there's plenty of ways if you're. In here at an are just in our discussion you understand there's a package for it right if you want to improve you want to do statistical forecasting you can find a package right that'll get you a little lift but. That's not really the power that you're going to unlock that next s-curve how you impact the business with analytics it really comes in with the data right so we undertook we went out there's plenty of. External data sources around automotive you take a Ford Focus we can figure out how many Ford float fahsai how many Ford folks are they're gonna build but we did some additional Legg working. And working with our and customers to collect data with them about where our finished goods go right and then you sprinkle in some secret sauce and what we ended up with is what we affectionately call the crown jewel of analytics at Worthington Industries and when I say that it worthington industry there's. Always look at fanfare of trumpets and confetti cannon kid cannons go off that's the crown jewel of analytics at Worthington and so then the question was well what do we do now so four years ago our then president of steel processing processing who's now our CEO Oh said. We got to figure. This out right that's great we had a big win we improved forecasts we carry less inventory we carry the right inventory flow through our factories are better and we're meeting customer service expectations okay what do we. Do now there's there has to be more opportunity go back to those sticky notes there's 10,000 sticky notes there has to be. Some strategically aligned opportunities for analytics I come on about four years ago and that was the task right this framework I'm not sure if you're familiar with Tom Davenport's work or the iia group a couple frameworks were in place there was a team that. Had been meeting a cross-functional team to talk about this and one of these frameworks that we started to use when we talked to the business when we talked to leadership or was about considering the questions. We're asking of our data right first would be do you have the data to answer your biggest business challenges right if you don't have it if we better figure out a. Way to collect it because without. It you're not gonna ask any questions of it because you don't have it but secondly we talked about let's think about the questions we're asking of our data and you can see across the columns there's the rearview mirror facing data hey every. Quarter we look at our sales and we see where there's some Delta against our goal we. Take action right but as you move out as you move down into the. Right we understand that similar to that automotive predictive forecasting model that's where. The value is right that's the point of competitive differentiation for us so we we put that in front of the the the leadership team and various levels of the organization but it was how do we start asking better questions more value added questions so we we partnered. With a consulting group that helped us on a extremely successful extremely expensive but extremely successful transformation of our business they came in came in senior leadership a quick overview of analytics and like any good consulting firm to scare the crap out of them you're so far behind everyone's doing it everyone's doing whatever maybe it wasn't digital then but. It was and it was successful because the ideas were flowing a lot of top secret ideas way. To blur this out but there were ideas and supply chain commercial and operations of how we can move the needle with analytics and so we did what any predictive analytics team of. Two people would do we had leadership sign us up for nine projects running concurrently bad. Thing was I Steve develops a bleeding ulcer loses like 15 years off his life positive part we drafted will Davis on the team or will was told voluntold that he would now be a member of the esteemed predictive analytics team at Worthington industry so one of those projects was. This price elasticity project and you can read there the business problem so this is about two and a half years ago you can read the business problem but in a nutshell its we get we quote business for our customers and what we did what we had was how. Many quotes that we went how many quotes did we lose how many quotes do we have in process at the end. Of the quarter at the end of a bidding season we could talk about how many quotes. We want how many quotes we lost and how long it took us to get those quotes out what we didn't know is pay for a specific region for a specific product are we getting what we should be getting right are we getting a premium for the product that we're processing no idea right what about our lost data. Oh wait that's on like. 42 different hard drives throughout the company when we lose we just sort of put it aside when we win there's a whole process that comes into place but we didn't have the lost data to even get to something as aspirational as a as a pricey. Elasticity model something else we throw out there. And this is a testament to the champion of this project and it was hey hey we could take all our financial data right when we win a quote and we process material we send it to our customer there's all sorts of. Financial data so for everything that we've won we can go through and look at hey we bought it for this we told the customer this. Here's how much money we made but that's not that's not you're not going to build a price elasticity model like that right if you go back to that six box we're still maybe on the top and over in the rearview mirror looking as far as the questions we're asking of that data and what he said was no we're. Going to change how we do this process right we're gonna disrupt our quoting group we're gonna find a way to collect this data very messy data I don't know how long the tables are forty fifty sixty. However many feels there are we're going to put processes in place to collect the right data because the end goal is a differentiator for us and that's what wills going to talk about as some of the steps that we've gone to answering that question and here are some the last thing I'll say about this before I turn over. Table I we did we did use that data we did use that financial data in my hands it was maybe a little bit better than worthless so this is a couple years ago when I first learned tableau I was in charge of this. Project why I don't know because there was three of us and hey you get supply chain you get commercial you get operations but what was fantastic it just a quick success story about tableau we put it in the hands of a subject matter expert someone in that. Group who had never touched her blow and it really revolutionized his work life he talks about how I would go to work. We'd run through quotes I learned. Tableau and he's absolutely energized he's a tremendous contributor to that group and you can see we've even he's branded some of these tools and production alized them in the quote. Group our scope group we call them but again if you go back to. That six box we're not going to get predictive prescriptive with tableau without some of the supercharging that will. Did right we've we're starting to cover that area the question is to really differentiate ourselves competitively with this data and these. Analyses we need to obviously ask better questions of our data and that's where I turn it over to my esteemed colleague mr. William Davis with 1l thanks yep my own can you guys hear me okay thanks so the reason Steve walks through that background is because how we got here then the some of the decisions we've made during the. Modeling process or a result of the backstory in the development of analytics. At Worthington and sort of the culture that surrounds in how we look at data so when you think about price elasticity. And a traditional sense you expect it as your price goes up your customers are. Going to buy less they're gonna have varying levels of sensitivity throughout that that range of prices we kind. Of knew that we're hoping. That our data would help us get a little bit more specific around quantitatively what does that mean and how can we start to use it on an individual quote by quote basis unfortunately that's what our data look like and really no amount of transformations or fancy math is going to get you know the points on. The right to resemble the curve on the left so we had to ask ourselves okay how are we going to deal with. Data that shape like this there's got to be other ways to cut it because we know inherently in economics that as demand goes up or as. Price goes up we expect our demand to fall so we know that our business has to follow that path the way we have the data now it doesn't look like. That so how do we peel it back how do we start to transform it to get it in a way that's going to show us what we know is there so to. Talk about that we first have to understand a little bit about our steel so for those of you that have never seen it that's what a steel coil looks like before it ends up in your seat or in your clutch plate or in the side of a. Combine it has a lot of attributes to it so if you think about a market that you're going to do price elasticity on you're gonna have some geographic components things that are specific to your customer or things that are specific to your product and where we face a challenge was in how. Diverse our products are and then that's sort of you know in the title of the session we say hi mix low volume it's because most of our coils of steel look very similar. But when you start to break it down there's a lot of different components that are sort of below what the eye can see that changes. Their applications changes the the value of what we can sell them for the first one is alloy which is essentially just a carbon content of the steel and in hundreds or even thousands of a percent sometimes and so whether there's a high amount of carbon or low amount of carbon can impact. Some of the different properties of it it impacts who's going to buy it and what they're going to use it for and therefore what price they're willing to pay we've also got some geometric properties like the thickness and the. Width of the. Steel so that not only impacts what people use it for but it impacts the equipment that we can use to produce it if there's if the they're specific enough not as many competitors are going to have the equipment that can do that that gives us a little bit of. A bit of an advantage and that too will change the shape of that curve there's other chemical elements in the steel again that the ability to have those or not have those is sometimes customer-specific and then lastly you've got the shape so it can come we can ship it to our customer this big coil sometimes we ship it. In in blanks that are further processing to other things in the Indus that may go you know in a wall or something like that so all of these properties just define the product you know it's not five or six SKUs. Like it might be for you know a widget a chair a table a product like that it's very customized for lack of a better word and so the problem becomes how do we define the market that we're gonna be in to look. At that elasticity where does one market stop and another market market start so the way that that reflects in the model and in our data set we only had about eight thousand quotes and I. Say only because in the world of big data eight thousand is you know you wait a second you've got 8,000 more data points in a lot of data streams in our case this was two. Or three years worth of our quoting history because quoting is. Infrequent and we had just started collecting that data so we faced a little bit of a challenge there and the challenge is exacerbated by the fact that because of all those product attributes. In addition to different characteristics of the market and the customers you've got a lot of variables and very few observations to spread those variables. Across so we did what any new analytics team that wants to make an impression would do is we built a big machine learning model and I say big I mean small but the result was it was extremely over fit we had you know the demand curve. That looks like that where a big you you're not gonna you know demand less as price goes up to a certain point then you're gonna turn around start buying more once you cross the threshold so we knew that we're ant we're. Running into a problem but in order to figure out. How to proceed we had to peel that problem back a little bit and so if we look at an example of one of the models outputs walking through a little bit of a decision tree we'd ask okay is this a cold-rolled product cold-rolled steel is a little bit more specialized it goes into some of the finer. Components of your. Car companies that can do well. There tend have a. Little bit of a competitive advantage so we thought it would be a good place to start to look at some differentiation so if it's cold road if it's yes okay is it is it a strip product which is just for the refinement if it is okay let's look at the price now well if it's under $50 we tend to. Win if it's over $50 okay what do we do next if it's under sixty we see that we lose but if we look at over sixty we see that we win and so now we found our problem is there's going to be confounding variables in. There which you know many people are going to understand that conceptually that there's other variables in there that the model hasn't captured. Or the more likely in our case we don't have enough observations of products or sales or quotes with those variables so how are we going to how are we going to account for this how are we going. To build a model to show us what we know is there we know that our. Customers have pricing and buying patterns and behaviors but right off the bat we don't have enough data to just throw it into a big model and get an answer back so our process forward was to say well we know that there are people in the business that do this every single day that's a little bit what Steve was. Talking about earlier we. Had this quoting process we were just starting to collect the data we were getting users in the business who were doing pricing every day engaged with tableau learning's have a little learning about analytics the company the business users saw the value in it. So then let's leverage them what's leverage their engagement more importantly their business knowledge and so the way we're going to do this is we're going to let them use tableau they can choose how the models. Built and then we'll build the model for them behind the scenes using R so this right here is the dashboard that that's sat in front of them during one of the early iterations and it's not going to be the. Prettiest one you see this week by it by any imagination though there's one or two sessions left today so maybe it'll be the prettiest one you see the rest of the day but the important thing is that the feedback we got is it was functional and as I. Walked through the different components it really helps the user understand the context of the model what the model is telling them why it's telling them that and how it fits into what they understand about their business so the first component of the model is the set of. Parameters that we have on the left side these are tableau parameters so there are things that we've given them the ability to choose or enter very specific values the first one is related to variance when we're looking at price. Elasticity we're fortunate we're in a market where there's a liquid financial market for our. All material you can go out into an exchange you can buy steel hot rolled coils futures it's not as cool. As Bitcoin but you can do it and there's a price for it and so when we're we're looking at what we're selling to our customers our customers aren't just interested in the sticker price that they see that how much are we varying from this underlying index because that underlying index is what drives a lot. Of contractual pricing across the industry and so with that we give the user the ability to say okay we're not going to model a price directly but. We're gonna model a variance from this underlying prevailing market price but do we want to use a very instance of. Percentage or a dollar amount because in. Times of high volatility that percentage can get pretty high in times of low volatility you may want to use a percentage rather than a dollar amount it just depends on the situation and again without enough examples it's tough for a model to pick up on that signal and that noise but a user who's quoting the business. Every day will have that knowledge to know when is it right to use a percentage versus an absolute value we then ask them to enter the current or the prevailing market price so that the model can take the historical data and. Normalize it because if you know the underlying index if it was at 400 last year and it's say 800 dollars now we need to know that so that. The quotes that were quoted at 400 can be sort of repriced. Into today's dollars before they used to train the model and then lastly we give them the ability to enter in a price of their own that they. Think hey this is the price that we think we should be charging the customer it may come. From some market intelligence maybe from an outside salesperson or just their their knowledge of the business but it puts what we what they think may be heuristic ly into the context of what the model is telling them the price should be and we'll see a little bit. Later on how that that is incorporated into the output so the things I walk through choosing the variance type that they want to use entering the market prey and then giving them the ability. To do a little bit of sensitivity analysis the second set of options we asked them to choose or that filters and this goes back to to what I was talking about before when you're trying to do the the elasticity calculation. Say okay we're going to do the elasticity for this particular market how do i define what this market is and as I was. Talking about that the features of steel they're choosing hey it's this product type this this deeper level product type may be the contract type has an influence if we've got enough data let's do this analysis for a specific customer our plants are geographically a little bit. Dispersed so by choosing the manufacturing location if it's a competitive product in that particular region then they may want to just look at historical data that comes out of that particular region and then the date so if we're in a. Period of fluctuating prices and they may want to narrow the amount of training data that they use so that we don't get noise from from periods that don't represent a climate that's similar to today but if things have been stable and they want the ability to include more data then they. Can open that price range back up and so. All of these filters are what has taken our historical set of data of quotes and and narrowing that down to only the quotes that they want to be training the model to help them make the decision for this new piece of business that they're quoting now what they see at the top is the historical. Data it shows the wins and losses based on all the parameters they've set all the quotes that meet those criteria these are all the quotes in the past. That fall into that particular category that particular market the winds are at the top the losses are at the bottom and you've got your price. Along the horizontal axis there there's a tooltip that gives them a little bit of information about each of those quotes they may if they see. An outlier they want to investigate it further they can take each quote has an identifier they can go look. It up and our quoting system if they want to know a little bit more about it but the the. Key component here is is why that chart or that plot is in the format that sin because we really have a binary variable either you won the quote. Or you lost so why are we showing it this way and the answer is to start to give the user an idea of the model that's going to be built underneath we're going. To be using the logistic regression model because we're trying to predict wins and losses and though we're not expecting the user to fully understand. How a logistic regression model works they may they probably understand how a linear regression model works from from a middle school or. High school class and we can build on that a little bit and say okay similar that we're going to draw another line through your data. Here this is probably what that line is going to look like it's going to kind of roughly curve from the top to the bottom and as you as you see that you know losses are zero percent winds are a hundred percent when. It comes to okay what's my probability of winning and then this line shows you what price points result in what percentage between 0 and 100 and so we we have our wins and losses and we use then price it's it's a univariate model and we predict what's the probability that we're going to win a quote at each price. Point along this continuum it looks similar to a demand curve which is the goal you typically see you know quantity and price we have percentage chance of winning versus price because we work in in a quoting type of process where instead of people coming in you know coming in and buying something off the shelf in varying quantities based. On the price we go through an RFQ process and generally it's all-or-nothing either win all the business or you lose all the business and one of the advantages we have is we have a little bit better grasp of those losses you know in the retail environment you may not know how many. People come in your store look at something and say I was gonna buy that but now I'm not and they walk out you don't have a good idea of that lost business we're fortunate that we do we don't have as many as much volume of data as people walking through stores and buying things but but we know. The other half of the equation and so that that allows us or causes us to use logistic regression model. To predict that probability and you can see a little bit of the code in the bottom. Left and I'll. Get into the detail of that a little bit later on but but for. The purposes of it now it's just we're running a logistic regression model to get our win probability as a function of price and we say okay that's great but then the question becomes well we don't want to win a hundred percent of our business because our price is probably too low and we don't want. To obviously lose a hundred percent because it's likely our price is too high but but you know. That's the Goldilocks question where is the right place on that continuum for us to price so that we're winning as much as we want or we're maximizing some measure of value to our company and so we introduced using the idea of expected value. Where you take your sales price multiplied your problem times your probability of winning and that gives you your expected value you know it's a little bit hypothetical you know if I do this quote a thousand times I quoted a hundred dollars I've got a seventy five percent chance of winning on an average I would end. Up with seventy five dollars and so that's how we've taken that percentage and say okay assume in a relatively simplistic world where should we set the price and that's what the graph on the. Top shows you is. How is that expected value changes your price changes and then we talked to our pricing analyst and say okay the math says that we want to price as close to the top of that curve as we can and with tableau. We add in the the reference line that shows where that maximum price is and then they can follow it down on the curve to see what the probability is and things like that lastly I talked about earlier they have the ability to enter in their own price for sensitivity analysis and what. That does is it draws a vertical reference line on there so they can see sort of the distance between okay maybe my customers. Telling me I need to be at a. Certain price for my sales guy is how is that compare to what the model says and then ultimately what's that what's our leakage from whatever the the theoretical economic maximum could be from that quote how much are we leaking by prices low and maybe that's okay maybe we want a price low because we've. Never quoted this company this customer before we're trying to get in the door for future business or we know that our quality hasn't been great as of late and we need to make sure. That we retain this business and so we're willing to price a little bit lower and those are the things that if you had you know an extremely large data set you could pick up on those signals but with a data set as small as ours we don't. Have the data to do that then we're going to rely on the user to understand that but we're going to give them the best model that we can with the data that we do have and then from there they can deviate from it as they see fit now obviously one of the risks with giving our. Users that much power without you know anybody looking over as they're training. The model because this model gets retrained every time a user selects a different set of parameters defines a different market and they don't have you know somebody from. The analytics team or data scientists looking over their shoulder saying that model is okay that one's not okay and so we had to put in a mechanism for doing that and I don't know there's a talk at. The end of the day yesterday and I forget the speakers from tableau that talked on a similar topic but. They talked about okay if you're gonna. Deliver this how do you give the user feedback on what's happening underneath and is it reliable because right now we're just pumping some colors and some lines out at them and they may trust that. They may not and if they get burned by it once are they going to come back so how do we build some of the trust in with our end users and it's relatively. Simple but we do it with this stoplight at Worthington we have a pretty strong culture of lean and. Continuous improvement one of the things they talked about being is you want your signals to your employees to be binary red or green win or loss easy to understand easy to know what. Action to take there's there's no ambiguity to it and so we give them a stoplight stop blades green if the model. Is good it's red if the model is bad well then the question. Becomes what's good and what's bad and when they hover over it they get a little tooltip that gives them a little bit more information so if it's good. We'll tell them the model is good or if it's bad will tell them it's bad we may explain why and then at the bottom we've included some of the parameters from the model that way if we need. To dig deeper if they need some help somebody from the analytics team can go over and support them we can see you know are there problems with some of the the coefficients this is the the p-value not as strong as we need it to be. So is is struggling from a statistical perspective or Cystic Allah valid but struggling from an economic perspective and what I mean from an economic perspective is we all have a kind of intuitive understanding of what's valid and what's not when it comes to demand curves and if we see a situation where the model. Showing hey is price goes up our probability of winning goes up then alarm bells start to ring in our. Head now again. Because of our our smaller data set that was a risk that we that we would run that and we've given the. User so much control they. May find a pocket of the market where we've raised prices. On our customers and because we control that. Market we do pretty well but now that data is going to show that hey you can just keep raising price as much as you want your demands going to keep. Going up so how do we look out for that and that's where we bring our and again we do a little bit of calculus and we can understand that the shape of that curve under the hood and look at if you look at just the first derivative of that curve to make sure that. It's always increasing. Or always decreasing and if you see that it's increasing it at any point throughout the curve then you know okay it looks. Like something here is going wrong maybe you know from a statistical perspective the p-value is low the accuracy was strong the coefficients you know the coefficients may have high p-values themselves but they're indicating that the curve is increasing and we know that that just doesn't. Correlate to a functioning market if you will and so the model is gonna say it's good mathematically but we need to walk them back and say hey economically we know this isn't valid and in that case the stoplight is going to turn red and they know that okay they need to go back and take a different look at. Some of the filters and the parameters that they've chosen so that's sort of the the front end or the the user-interface side of the model then after that after this we'll get into a little bit of. What's going on under the hood and why. We design it that way but what made the journey successful and how did we take you know of is it's not the prettiest vis that's got a lot of math going on under the hood I get people to buy in and and you have to make sure that you're solving a real. Problem that they have they know that they're collecting this data they. Know that there are times that we win when it doesn't make sense that we lose and it doesn't make sense customers are always going to tell you to. Because of price your sales guys especially always going to tell you that it's. Because of price but do you have the data to back that up and do you have the data to say you're telling me it's price but if I look at what's happened in the past and. What's happening now in similar markets I can tell you that it's not price and so making sure that you're solving a real problem so the business is engaged and it's a tool that if it breaks are you gonna get a call as somebody that I call you and say I can't do my job because the tool that you. Built for me is broken and and if nobody makes that. Call when it breaks then you have to take a step back and say are you solving one of your biggest problems or not you have to be aligned so the using our stand and standing up a tableau server that connected to our all that happened at the same time as. We were trying to deploy this model so we were getting folks on the IT side having to stand up hardware for us on the analytics team we were integrating the two at the same time and then we're. Trying to get our salespeople our pricing people trained up on tableau getting them access to tableau server and and making sure that they not only knew how to use tableau but how do they use the model that we've built for them and so alignment across all three groups is important because if you are solving one of. Your biggest challenges then everybody is going to be on board working towards that same common goal you want to introduce complexity and pieces you know Steve showed you two or three iterations of what the tool looked like before this this wasn't the first time these folks. Saw something related to their pricing process it's just the latest in a continuum and we've got further iterations and. What you've seen up here so if we try to throw it all on them at once then they're gonna have no context for why are we giving this to them what do we expect them and do with it I mean going along with that is allowing. Users to see their data so in our case they never seen that data before that win in lost history as Steve said we just started collecting it so we showed it to them in that top plot. Where they could see their wins and losses. And how they won lost across prices but we showed it to them in a way that helped them understand how the model is functioning other underneath and so they can see hey this quote I remember doing this quote they can you can tie the pieces of data that they're seeing on the screen and in their models tie it. To something that they're experiencing and doing in their job every single day it so I'm going to close with walking through a little bit more of the technology stack and how it's built and why we chose to. Build it that way and part of it selfishly is because as we're going through building it I didn't find. A whole lot in the tableau community and I'd like to start talking about it and I'd like to hear other people start talking about it so we you know if we can build to understand develop best practices for for how this integration is going to work because from like I said from my experience not. As many people were using it as I'd hoped or at. Least from what I could find through the community forums so this is what our whole stack looks like. End to end I'm just going to walk through the pieces individually so me as a data scientist I build the models in our and I actually I don't put them directly into tableau I push them into. Our version control system and I'll talk a little bit about why that is in a second but the the version control system takes the models that I've built and they're built inside of an r package and the reason they're built inside of an r package is a couple things first there's automated testing built into those are packages you. Can use packages like test that to make sure that that everything's building and running correctly and as you expect so that when it gets out there in front of your users it's not going to. Break and if you're putting the code directly into just their tableau calculated field you're not going to get that it's going to break right in front of them they're not going to know what to do and you're going to lose that engagement you know the whole package infrastructure also comes with some built-in documentation which I think. In what I've seen a lot of times gets neglected both on the data science side as well as on the tableau side putting tools and visualizations out there that are meant to be used interactively but not providing documentation on what do these fields mean how am I supposed to use that what am I supposed to click on it. So at least from from a modeling perspective using R in that package environment builds in some of that documentation so once that package is built it gets pushed we have an internal cran mirror because of security reasons our IT group doesn't like us going out getting packages open source is new to a lot of organizations. So they actually just mirror every single night cran inside of our firewall and then we build our package to that mirror so that if somebody wants to develop off of our package inside the organization or if another. Application needs to go and get it like our serve it's available where all the other packages are going to come from so from there every night we have scripts that run that from our our serve server that goes out and checks to see if that package has been updated so when we push updates to it. They automatically flow through we. Do in the middle of the night so we can restart that our. Serve process without interrupting any of the business users and then from there that the package gets called from inside tableau and so here's what that looks like a little bit more deeply inside tableau the the screenshot on the right is what you saw on the first slide where all we're doing. Is we're calling the library and the custom function that we wrote and then the the screenshot on the left is what that function actually looks like under the hood and I think the biggest convenience at. Least from the data science development perspective of building it in this manner is that when you want to make a change if you want to introduce maybe you want to introduce some type of sampling methodology you know. If we lose 70%. Of our business or our quotes and win 30% you're gonna have a pretty imbalanced training data set there so maybe you need to do some down sampling or some up. Sampling that's something that you know enhancement that you want to release you'd have to go out to tableau server. If you're using a data source that's on the server. And an extract you're gonna have to download that workbook you're gonna have to download the data source you're gonna have to edit the calculation on your desktop and. If you're using that calculation in more than one place so if I'm not only pulling that the predictions out of the model but I'm pulling maybe the p-value I'm pulling one of the coefficients so it's duplicated in three places I'm gonna have. To go make that change three times and I'm going to save that workbook I'm going to reiax tracked the data. Source I'm going to push it all back up to my server and in this case you have one output object from the model in our case it's that wind prop but you could go further and you could you know what we've done is when we're going to extract that p-value as well as. The prediction we actually concatenate them together into one vector from R. And return that vector to tableau and then inside tableau we parse that vector because it allows you to leave. That calculation the same in tableau and make all the changes in R you can flow through the entire version control the entire testing process that goes behind the scenes with our serve without. Having a touch tableau you can change the model type you can change any of the pre-processing that you do and all of it's done in an environment that's a little bit more aligned with data science and then tableau doesn't know any different it's just running the same calculation from the same package but it's getting. The updated results and so why I build this infrastructure the inefficiency of continuous integration and continuous deployment we could push those updates without any having to do any work on the tableau side which makes it a lot easier for our users we're not having to take those those workbooks down. Or anything and sometimes it can be a lot faster than having to download the workbooks stability and reproducibly reproducibility of version control and I know the version control is just sort of starting to ramp up on the on the data science side in a lot. Of cases but it's really helpful especially on a small team like ours where if I'm island vacation somebody needs to go in and make a change it's easy enough. For us to work across one another automated testing I think this is really big one of the easiest ways and there's a session yesterday that someone alluded into it one of the easiest ways to lose engagement especially with a tool that is a little bit a little bit higher performing or with a little bit more of things. And events going on to help it render the easiest way to lose engagement is for one of those things to break and so when you have that built in testing the likelihood of those things. Breaking obviously it's a lot smaller and that's a huge advantage. To retaining your users the performance games so by keeping the are serve server and the tableau server separate it gives you some performance gains they're not competing for resources and often times. The it's easier to support concurrency if you. Have multiple users hitting that sit that workbook at the same time keeping them separate it generally seems to perform better especially if you keep that our serve environment on a linux box then spinning up new processes is a lot smoother without not only competing for resources but sometimes memory and variables getting crossed as well if people. Are using it at the same time and lastly the security this was actually the first big hurdle this is what led to the thought. Process for developing that entire stack is when our IT department said we need that that Cranmer to be inside of our firewall and. We got to thinking okay if we're standing that up then then what is the entire process look like how do we how do we build it out further if we're. Going through all this work what's really the best way that we can do this but you get that security of making sure that and it doesn't have to be the entire cran mirror it. Could be just your package if that's all that you need it you know in the odd situation doesn't have dependencies it doesn't have to be a full near there are plenty of our packages to let you stand up. In a mirror that's just what you put into it but again that provides a lot of infrastructure on testing documentation etcetera that you don't get elsewhere so that was all that we had you know if you guys could fill out the survey that'd. Be great and we'd be happy.

 


Worthington Industries Company News

Wed, 08 May 2019 19:45:17 GMT
Looking At Worthington Industries, Inc. (NYSE:WOR) From All Angles - Simply Wall St
Looking At Worthington Industries, Inc. (NYSE:WOR) From All Angles Simply Wall St Want to participate in a short research study? Help shape the future of investing tools and you could win a $250 gift card! Worthington Industries, Inc.
Thu, 25 Apr 2019 07:00:00 GMT
Worthington Industries, Inc. (NYSE:WOR) Delivered A Better ROE Than Its Industry - Yahoo Finance
Worthington Industries, Inc. (NYSE:WOR) Delivered A Better ROE Than Its Industry Yahoo Finance Many investors are still learning about the various metrics that can be useful when analysing a stock. This article is for those who would like to learn about ...
Tue, 30 Apr 2019 17:27:16 GMT
Is There An Opportunity With Worthington Industries, Inc.’s (NYSE:WOR) 28% Undervaluation? - Simply Wall St
Is There An Opportunity With Worthington Industries, Inc.’s (NYSE:WOR) 28% Undervaluation? Simply Wall St Want to participate in a short research study? Help shape the future of investing tools and you could win a $250 gift card! Does the April share price for ...
Wed, 01 May 2019 13:00:00 GMT
Worthington Announces New Presidents NYSE:WOR - GlobeNewswire
Worthington Announces New Presidents NYSE:WOR GlobeNewswire COLUMBUS, Ohio, May 01, 2019 (GLOBE NEWSWIRE) -- Worthington Industries, Inc. (NYSE: WOR) today announced that Jeff Klingler has been named ...
Sun, 21 Apr 2019 07:00:00 GMT
Worthington Industries $WOR Technical Update - Stock Traders Daily
Worthington Industries $WOR Technical Update Stock Traders Daily The Worthington Industries (NYSE: WOR) update and the technical summary table below can help you manage risk and optimize returns. We have day, swing, ...
Thu, 09 May 2019 13:14:22 GMT
Global Stem Cells Cryopreservation Equipments Market 2019 Competitive Analysis – Chart, Worthington Industries, Cesca Therapeutics, Shengjie Cryogenic Equipment - Canyon Tribune
Global Stem Cells Cryopreservation Equipments Market 2019 Competitive Analysis – Chart, Worthington Industries, Cesca Therapeutics, Shengjie Cryogenic Equipment Canyon Tribune MRInsights.biz published latest research on Global Stem Cells Cryopreservation Equipments Market Research Report 2019 study major consideration after ...
Wed, 08 May 2019 17:21:00 GMT
Watch List Basic Materials Stock Buzz: Worthington Industries (WOR) - Stock Digest
Watch List Basic Materials Stock Buzz: Worthington Industries (WOR) Stock Digest Worthington Industries (WOR):. Checking on recent price levels compared to prior highs and lows, we have seen that Worthington Industries (WOR) recently ...
Mon, 06 May 2019 07:49:36 GMT
Biobanking Market Top Key Operating Vendors Brooks Automation Inc., Worthington Industries, Modul-Bio, LabVantage Solutions. Inc, Tecan Trading AG, TTP Labtech, Excilone SARL, Ziath Ltd, among others - StockNewsMagazine
Biobanking Market Top Key Operating Vendors Brooks Automation Inc., Worthington Industries, Modul-Bio, LabVantage Solutions. Inc, Tecan Trading AG, TTP Labtech, Excilone SARL, Ziath Ltd, among others StockNewsMagazine All the information supplied via this report properly gives explanation of various facts and figures to the business. The market share of major competitors on ...
Thu, 18 Apr 2019 07:00:00 GMT
Where The Dangers Lie And Where To Look For: Worthington Industries, Inc. (WOR), KLX Energy Services Holdings, Inc. (KLXE) - GV Times
Where The Dangers Lie And Where To Look For: Worthington Industries, Inc. (WOR), KLX Energy Services Holdings, Inc. (KLXE) GV Times The price of Worthington Industries, Inc. (NYSE:WOR) went up by $0.34 now trading at $40.56. Their shares witnessed a 29.09% increase from the 52-week low ...
Thu, 11 Apr 2019 07:00:00 GMT
Present Performance: Worthington Industries (NYSE: WOR) - TNN
Present Performance: Worthington Industries (NYSE: WOR) TNN Worthington Industries is in the steel industry and is part of the basic materials sector. The company CEO is John P. Mcconnell. Worthington Industries Inc is ...