Posts

TFS & GO & Chef, oh my: Onboarding update!

Well, as of yesterday afternoon, we have one website and one database actively onboarded into our automation system.  Unbelieveable amount of work, but totally worth it. What was chosen as the candidate for this pilot is our primary web front-end (referred to as WebUI here on out) and (one of) it's accompanying databases ('the database').  I'll try and describe what we're doing and how it's getting accomplished. Pipeline overview ...as of today, anyways... Database (pipeline group) Database Build (pipeline) Builds the SQL scripts - we have pre/deploy/post scripts - pre/post are data transformation, 'deploy' is functionality-related.  Dev opted to use the VS equivalent of SQL compare - we will be using the Prod database as the true reference.  The 'deploy' scripts are automatically generated - pre/post scripts are done by hand and are optional. Database Deploy (pipeline) Snapshot script - 'reverts' the snapshot, takes anothe...

Elasticsearch - Curator & update#2

Initially when we started looking at ES I had grand visions of months and months of accumulated data.  Unfortunately, you need a big hardware hammer to deal with that amount of data, so after living with 60 days of log data and having 5 nodes (4GB heap each, 8GB RAM) constantly piling up to 80-90% heap usage, we decided to drop back to 30 days (still adequate for what we're doing today). I would caveat that statement with 'well, that's what I know today'.  For all I know, it's supposed to be running at 80-90% heap usage.  If that is the case, seems strange that tools like 'kopf' show red at 80% heap usage, however - and ES docs indicate that anything past 90% is BadNewsBears. Curator Also re-read the Curator docs after seeing that optimizing helps get a little bit more performance out of the cluster, and sure enough there is an 'optimize' switch. Our cron job list now looks like this: 20 0 * * * /usr/bin/curator --host camasves01 close --ol...

Dashboard Rotator

First - this is 100% worth the $20.  I submitted a question and within 72 hours he had written a new feature in, plus commented on a solution to a situational problem I was having.  Well done, sir. Second, this is a voluntary plug; I'm just a happy customer. We have a glorified PC that has two big ol' Matrox cards, currently powering five 50" TVs around the office via HDMI-ethernet converters.  It's not a great 'video server' solution, but it works (you can't see what's on the remote TVs from in the server room...so WindowsKey+arrows!). Anyways, each department has their own TV and I'd like to start rotating pages.  We're using Grafana which has a playlist feature, but that's limited to ' on the playlist, not on the playlist ' functionality.  We have multiple TVs that need different sets of pages displayed.  Furthermore, we have a pile of other pages to display (simple-nagios-dashboard, kopf ES plugin, Kibana dashboards, etc) th...

Newbie datacenter lesson: The implications of 120v and 220v

So I've never initiated a fresh colo build before, and it's showing.  And before you get all pedantic, I know 220v is probably not the right term (208, 240, etc), but it's what I'm using.  So there.  Newbie, right? Problem: Not getting super specific about what our power circuit will be. Assumptions by both parties caused this, but the fault really only matters on my side.  It ended up being that I ordered the wrong PDUs (well, right if you assumed 220v), and moving to a 220v circuit would have been an additional monthly cost.  I incorrectly assumed that we would be getting a 220v/20A circuit. The difference between 120v/20A and 220v/20A is significant The difference between L5-20 and L6-20 is significant The difference between 20A and 30A is significant Let's address these, because it's been a heck of a learning experience for me. 120v and 220v Why is it a big deal?  To sum it up really quickly - 220v basically means you can house twice the...

Buying storage? Don't forget the cables...

It seems silly, but I'm writing this down because 'fool me twice' and all that. So in the past year or so we've purchased two EqualLogic SANs from Dell.  Both times the reps (different reps...but that is another story) forgot to talk about/sell me cables (read: I forgot to ask ), so we got a nice shiny SAN that we couldn't plug into anything.  You would think that I'd have learned from the first time...thankfully we had some spares, so wasn't a huge deal this time around, but it did mean this: I have an 0.5m cable length requirement, and a 3m cable.  Now this  is silly.  What's doubly annoying is that the impact on the budget, while not huge, is not insignificant (mostly because, you know...it wasn't included IN the budget).  Now that we're all 10gb storage network, the SFP+ Twinax cabling is pricey, even from non-OEM sources.  The 0.5m cable I need is $90 a pop! (MSRP, anyways) There you go.  Buying storage?  The good ol' days of ...

DFSR - eventid 4312 - replication just won't work

This warning isn't documented that well on the googles, so here's some google fodder: You are trying to set up replication for a DFS folder (no existing replication) Source server is 2008R2, 'branch office' server is 2012R2 (I'm moving all our infra to 2012R2) You have no issues getting replication configured You see the DFSR folders get created on the other end, but nothing stages Finally you get EventID 4312: The DFS Replication service failed to get folder information when walking the file system on a journal wrap or loss recovery due to repeated sharing violations encountered on a folder. The service cannot replicate the folder and files in that folder until the sharing violation is resolved.    Additional Information:  Folder: F:\Users$\user.name\Desktop\Random Folder Name\  Replicated Folder Root: F:\Users$  File ID: {00000000-0000-0000-0000-000000000000}-v0  Replicated Folder Name: Users  Replicated Folder ID: 33F0449D...

Add an RDM, get an error! (EqualLogic)

Google fodder. I was able to reproduce as follows: Take a VM that's already working, shut it down (in my case, 2012 R2). Add an RDM disk (using a new controller or not, both gave the error) Try to power up the VM. You will get an error: 38 (Function not implemented)  Goes on to talk about 'cannot open virtual disk or one of the snapshot disks it depends on' - even though no snapshots. Thankfully, even though the Google hits were few, I figured it out.  The issue? This is for moving our physical Veeam backup server into a VM.  For the B2D target I wanted to use an RDM that is on a different SAN than the VMs.  When I created the volume in the EqualLogic group manager I had the option of 512-byte sectors, or 4k sectors.  Figured, hey, 2012R2 can deal with 4k! Maybe it can, but ESXi 5.5.0 (update 2b?) sure didn't. I nuked the volume, recreated it with 512-byte sectors and it works no problem. FWIW, the array is running the latest firmware (at ti...