Skip to main content

Posts

Showing posts from February, 2015

TFS & GO & Chef, oh my: Onboarding update!

Well, as of yesterday afternoon, we have one website and one database actively onboarded into our automation system.  Unbelieveable amount of work, but totally worth it.

What was chosen as the candidate for this pilot is our primary web front-end (referred to as WebUI here on out) and (one of) it's accompanying databases ('the database').  I'll try and describe what we're doing and how it's getting accomplished.
Pipeline overview ...as of today, anyways...
Database (pipeline group)Database Build (pipeline)Builds the SQL scripts - we have pre/deploy/post scripts - pre/post are data transformation, 'deploy' is functionality-related.  Dev opted to use the VS equivalent of SQL compare - we will be using the Prod database as the true reference.  The 'deploy' scripts are automatically generated - pre/post scripts are done by hand and are optional.Database Deploy (pipeline)Snapshot script - 'reverts' the snapshot, takes another snapshot - after …

Elasticsearch - Curator & update#2

Initially when we started looking at ES I had grand visions of months and months of accumulated data.  Unfortunately, you need a big hardware hammer to deal with that amount of data, so after living with 60 days of log data and having 5 nodes (4GB heap each, 8GB RAM) constantly piling up to 80-90% heap usage, we decided to drop back to 30 days (still adequate for what we're doing today).

I would caveat that statement with 'well, that's what I know today'.  For all I know, it's supposed to be running at 80-90% heap usage.  If that is the case, seems strange that tools like 'kopf' show red at 80% heap usage, however - and ES docs indicate that anything past 90% is BadNewsBears.

Curator Also re-read the Curator docs after seeing that optimizing helps get a little bit more performance out of the cluster, and sure enough there is an 'optimize' switch.

Our cron job list now looks like this:
20 0 * * * /usr/bin/curator --host camasves01 close --older-than …

Dashboard Rotator

First - this is 100% worth the $20.  I submitted a question and within 72 hours he had written a new feature in, plus commented on a solution to a situational problem I was having.  Well done, sir.

Second, this is a voluntary plug; I'm just a happy customer.

We have a glorified PC that has two big ol' Matrox cards, currently powering five 50" TVs around the office via HDMI-ethernet converters.  It's not a great 'video server' solution, but it works (you can't see what's on the remote TVs from in the server room...so WindowsKey+arrows!).

Anyways, each department has their own TV and I'd like to start rotating pages.  We're using Grafana which has a playlist feature, but that's limited to 'on the playlist, not on the playlist' functionality.  We have multiple TVs that need different sets of pages displayed.  Furthermore, we have a pile of other pages to display (simple-nagios-dashboard, kopf ES plugin, Kibana dashboards, etc) that are …

Newbie datacenter lesson: The implications of 120v and 220v

So I've never initiated a fresh colo build before, and it's showing.  And before you get all pedantic, I know 220v is probably not the right term (208, 240, etc), but it's what I'm using.  So there.  Newbie, right?

Problem: Not getting super specific about what our power circuit will be.

Assumptions by both parties caused this, but the fault really only matters on my side.  It ended up being that I ordered the wrong PDUs (well, right if you assumed 220v), and moving to a 220v circuit would have been an additional monthly cost.  I incorrectly assumed that we would be getting a 220v/20A circuit.
The difference between 120v/20A and 220v/20A is significantThe difference between L5-20 and L6-20 is significantThe difference between 20A and 30A is significant Let's address these, because it's been a heck of a learning experience for me. 120v and 220v Why is it a big deal?  To sum it up really quickly - 220v basically means you can house twice the equipment.   Amps = Watt…

Buying storage? Don't forget the cables...

It seems silly, but I'm writing this down because 'fool me twice' and all that.

So in the past year or so we've purchased two EqualLogic SANs from Dell.  Both times the reps (different reps...but that is another story) forgot to talk about/sell me cables (read: I forgot to ask), so we got a nice shiny SAN that we couldn't plug into anything.  You would think that I'd have learned from the first time...thankfully we had some spares, so wasn't a huge deal this time around, but it did mean this:

I have an 0.5m cable length requirement, and a 3m cable.  Now this is silly.  What's doubly annoying is that the impact on the budget, while not huge, is not insignificant (mostly because, you know...it wasn't included IN the budget).  Now that we're all 10gb storage network, the SFP+ Twinax cabling is pricey, even from non-OEM sources.  The 0.5m cable I need is $90 a pop! (MSRP, anyways)

There you go.  Buying storage?  The good ol' days of 'have et…