World of EUC

2003058516.jpeg

dang, that would be perfect to hide in my little wiring panel

"Ubiquiti Networks Early Access store is currently available only for US and Canadian-based customers."

<https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Early-Access>

It’s sold out anyways. Beta stuff goes quick. Sometimes they restock it.

<https://www.gizmochina.com/2019/11/15/xiaomi-29-inch-curved-gaming-monitor-coming-soon/>

<@ULJ3D468M> what was that PCIx riser card you were saying worked well with the E200-8D?

<@UKJA0MZN1> Confused, you're asking about the PCIx riser card and soon after you have it running :wink: I have it running as well but MDT deployment breaks at 37% due to over heat at 104 degree C with 3 Noctua fans in Fan Mode = Full. Only CVM is on this host. I have it in a rack though, so will try to take it out to investigate further.

I always ask if it can be done as I try to do it. I’m impatient :rolling_on_the_floor_laughing:

So looks like you can do UEFI for the VM so the CE must be able to install in UEFI hardware no issue yes?  Saw you me twitter post so now I’m really curious. Had to give back my lab hardware so going to be buying new soon. 

I think Nutanix has always installed on UEFI, but UEFI support in VM came recently in the latest release

Not sure about production software but the last CE wouldn’t. <https://www.icrill.com/post/nutanix-ce-a-truly-community-edition>

Okay, yeah you probably right, I did not change anything in BIOS so likely.

<@ULJ3D468M> I installed all nvme on my NUC because of the two M.2 ports. Since CE had that bug I went with SSD on my E200-8D and used a sata port for one of the drives.

Check this blog post Eric. <https://twitter.com/dchung615/status/1199819179395866625?s=21>

Awesome, yeah the nodes (both VMW are NTX) are really living on the edge

Did you have overheating on ESXi with nvme?

Yeah, the production node with all the VM's running is close to 100, at 104 it just blacks out. So ordered 4 urgently :wink: <https://twitter.com/xenappblog/status/1200069625482022917?s=20>

Up until now I've just ignored the CPU warnings, time to take action before its to late.

Forgive me, been a while since I've had to do this. Is the proper upgrade order AOS then AHV? Don't have prism central deployed in the CE lab.

Yes that’s what I do on my prod nodes as usually the updated AHV doesn’t show up until AOS is upgraded. However with CE I’ve not had good luck with upgrades. Yesterday on another node I tried the upgrade. AOS took. AHV didn’t.

<@UKJA0MZN1> now you understand my Black Friday style shopping during Synergy :-) you should grab that fan as well, just sayin

ehh, AOS upgrade is behaving similarly to previous upgrades where I have to wait for CVMs to report back as 100% then manually reboot them once more to get them to fully start talking to the cluster again

image.png

AHV naming convention is a little weird in the update

ahv upgrade seems to just be stalled after entering maintenance mode. sigh

[root@NTNX-COLO-B ~]# virsh start NTNX-e4ede571-A-CVM
error: Failed to start domain NTNX-e4ede571-A-CVM
error: internal error: qemu unexpectedly closed the monitor: 2019-11-28T18:11:03.181406Z qemu-kvm: Warning: Large machine and max_ram_below_4g (536870912) not a multiple of 1G; possible bad performance.
2019-11-28T18:11:03.243991Z qemu-kvm: cannot set up guest memory 'pc.ram': Cannot allocate memory

[root@NTNX-COLO-B ~]#

Image from iOS.jpg

<https://vcloudone.com/2019/11/cpu-cooling-for-supermicro-e200-8d-and-e300-8d>

Unifi AP AC Pro and LR WiFi speeds? I'm just wondering, what speeds do you get on SMB trafic from a Windows machine? I get 60MBps max. (devices have intel ac 9260) when using this config on my 5G Radio: Channel Width - VHT80, Channel 124(DFS) and Transmit Power on High. I don't have that many WiFi networks near my home, no active 2.4Ghz nor strange behaving items like a Chromecast. Is this 60MBps bad? If so, what configs do you have on your 5G radios?

Its also great if you e.g. have only one PoE cable in the seeling and want to add a camera side by side your AP

Very true, I just love their kit. It's easy to setup and performant.

That's a lot of bang for your buck.  I was expecting it to cost several hundred.

Their kit is honestly affordable and really good. Well presented, performant. For what you pay it's honestly brilliant stuff.

Looking to spend 1500$ on a lab. Need CPU and RAM with some NAS. Any ideas?

What do you want to do with it Ray, what sort of technologies do you want to play with? Any switching required?

<@ULLH5JGFR> I am in the middle of building an AMD based system. 8c/16t, 32GB, SSD storage. I spent about $500 so far.

My recommendations are if you're looking to mostly lab at the app layer, a single beefy system is the way to go. If you want to replicate a particular hardware aspect to learn it (storage, network, etc.) then breaking it out is best.

Vmware with full citrix environment, with options for mscs for ca. Things like that. 64 gb of ram to start with atleast

I think you could do a small version of that on a lower tier desktop. For more I'd look at workstation/HEDT or full on server.

It might be important. Citrix ADC is not supported with AMD CPUs. I have tested multiple builds, and all are unstable and produce multiple core dumps. It's also mentioned here:
<https://www.citrix.com/products/citrix-adc/resources/netscaler-vpx.html>

I have a small lab. But i need something as close to a enterprises solution. But without breaking my wallet lol

Yeah,same here. I have a AMD Ryzen setup, I love all of my CPU cores  :smile:

Hmm, well I have another box I can probably run the ADC on.

new intel i9-10980xe processor, 18 cores, $1k dollars. I am building a new computer for lab at home, 128gb ram was another $550.

samsung evo plus 1tb, same perf as Pro, but half the endurance and almost half the price.

That's cool. My only thing is local storage, I can't do rdm for ca file server. That's why I'm looking for a nas.

That's extremely disappointing though. I don't see why that would matter to them.

Same here...just thought I would mention it, since Citrix is only mentioning it in the very fine print

Jarian is going to reprimand us now, for not threading :smile:

E-200 8D mate. Great for a lab. Plenty of grunt to get you going and flexible 

In my mind it's better to have multiple lower end nodes to get more cores as storage and memory are cheap.

All my stuff is supermicro, comes with ipmi, remote console etc...

:point_up_2::skin-tone-2:Dave, please. David is like my mum telling me off :joy:

I have ubiquiti edge switches for routing

<@ULHV2NMGQ> lol, that's hilarious. I'll remember that ;-)

But agree. IPMI is a huge bonus. Also have UniFi kit in lab and at home. Makes segregation of lab networks easy but still visible via the cloud key

I've now switched everything in the house to ubiquiti. Its so good for the price point.

I liked Ubiquiti, but they EoS'd my AP after only a few years.

Yup. With blab trunking and bonding ports as well you get a decent throughput lab side and a solid network “home” side

That's true. Support is shorter. I had that with an enterprise client.

Obviously Dave lol. I was pissed though to find my ERPoE5 only carries VLANs. VLANs are defined via sub interfaces but you cannot assign the individual ports.

<@ULLH5JGFR> I picked up a brocade nfv appliance which is actually a low spec'ed E300-8D. It was £500, I just had to get an NVMe drive and more ram. For about 800 total I got a 1tb NVMe and 32gb ram.

Yea I'm going to start looking into these guys.

E200-8D. If you replace stock fans with 3x Noctua you’ll also need a CPU fan so you don’t burn it. Goes for all hypervisors. Small box, lots of heat, but silent as a lam.

<https://www.amazon.com/Supermicro-SYS-E200-8D-6-Core-2x10GbE-Server/dp/B06Y2PW3RB/ref=mp_s_a_1_3?keywords=E200-8D&amp;qid=1577150928&amp;sr=8-3|https://www.amazon.com/Supermicro-SYS-E200-8D-6-Core-2x10GbE-Server/dp/B06Y2PW3RB/ref=mp_s_a_1_3?keywords=E200-8D&amp;qid=1577150928&amp;sr=8-3>

It's good but.... You'll want an NVMe drive to take full advantage. That drive is an SSD , 500MBs.

That's a pretty good deal Ray. I'm struggling to put parts together for the same price.

Yeah want NVMe, my IaC deployment takes 45 min on NVMe &amp; more than 2 hours on SSD. Also that model have 2x 16GB, you should go 2x 32GB

NVMe, ok u told me that my bad. They get pricey lol

Ok so NVMe and replace the fans. Leee helped me discover your framework automation. This whole time I'm building manual. 🥴. Eric its awesome you do this stuff.

<@ULKDZHKFY> - what’s the best place to pick up a e200 in the uk. Everywhere I look they are sold out. :cry:

Popular bits of kit. You ever had any dealings with Boston UK? They are the SM disti in the UK. I have an account with them for the business. If you give me the spec you want I'll sort a quote for you mate.

Perfect mate. I’ll send one over later if that’s ok?

Essentially I just wanna e200-8d with a 512 gb m2 / 128gb ram and a 1tb SSD if you have a canned quote

I don't have a canned quite as all of mine are E300 units. I will sort you one out though.

Sent through mate. As soon as I hear I'll let you know.

<@ULL6F7DRA> I was looking into the Intel requirement for VPX, and I'm not seeing that reflected in documentation.

Only thing I can think of is the AES-NI piece, but it doesn't look like it is required.

Hey <@URV4HE1LL> sure thing. It's mentioned in the "fine prints" during the end of this article:
<https://www.citrix.com/products/citrix-adc/resources/netscaler-vpx.html>

Yeah, it is funny and also a bit strange why Intel CPUs are the only supported CPUs.

<@ULL6F7DRA> So I went ahead and deployed the OVF in my lab, on a Ryzen 7 2700. It deployed just fine, and seems to be running. I still need to finish up configuration and get things going. However, a colleague mentioned that Storefront isn't supported on the free version, so this might be a fool's errand as I'm setting up a CVAD lab.

Yeah, that's what I see as well. But in my lab I see some random core dumps every 10 to 15 minutes and eventually the Netscaler runs out of disk space because of the huge amount of core dump logs

What's odd is it's essentially Linux, which supports these just fine. With the mentions of AES-NI I wonder if it's continually trying to utilize it and running into trouble when it's not found.

That might be. I have not been able to find any information regarding AMD CPU support or issues with AMD CPUs in Netscaler. However considering that AMD is now entering the server CPU market again, with their EPYC CPUs, we might start seeing more people running Netscaler with AMD CPUs.

Ray, check these babies out SM E200-8D or E300-8D

Yup, really nice but I don't like that mix of ports

It can do 2.5/5G as well, that's pretty slick for that price range.

Thanks. Now that the lab is fully populated, it is getting quite warm in there. The work the original HVAC contractor did (who is now out of business) is useless. The HVAC guy who came out today wants to recommend a mini-split, but he requested all the BTU info for all the devices. That came out to a max BTU per hour of 11,295.40.

We have one in our rental unit. It's very quiet, I think around 45 db. The external unit is also very quiet. We have a Daikin, but there are a number of reputable manufacturers like Mitsubishi, for example.

I think ours is rated at something like 20k BTUs (it's a 3 ton capacity unit).

I have two Mitsubishi in my office. I don’t notice them at all. I do notice the beverage cooler though if that helps.

If you are using your mini-split to cool your "server room", is your mini-split a residential unit or commercial unit with low ambient temperature?

My HVAC contractor is recommending a commercial unit which is almost twice the cost of a residential unit. The difference is the commercial unit will still cool even when outside or attic temps are below 20F. In TN, that is maybe three weeks a year

Our unit works fine here (a Daikin) even if it's around 0 F. I'd pass on a commercial unit. That said, we generally only use it for A/C and not for heating.

We went with this configuration:
18,000 btu "DIAKIN"17 seer ductless heat pump system
"DIAKIN" Indoor wall mounted air handeler
"DIAKIN" wall mounted outdoor condenser

Including installation and all plumbing and electrical work, it came to around $4,300.

The HVAC contractor came back today and gave me two quotes. One for a residential unit and a commercial unit. There is a lot of electrical and plumbing work required for either unit. The difference in price is $1,417. Both units are 12000 BTU. Price also includes undoing all the work the previous HVAC contractor (now out of business) did in the "server room".

12000 BTU should be enough I would think, given the scale difference between your application and mine. Cannot understand what the extra $1400 gets you other than maybe a longer warranty? On the Daikin, which we went with even though more expensive than the Mitsubishi, it was 10 years vs. I believe 3, which seemed worth it.

Both units have a ten year warranty. The difference is the ambient temperature guarantee to provide cooling in very cold outside temps

Ah, OK, thanks for the clarification. Maybe our alternative vendor wasn't Mitsubishi, I can't recall offhand.

For lower temperatures the setup needs to have "Low Ambient Temperature" in its description. Up here in Canada, that's mandatory. :wink:

20F is around -7C or so. So, Low Ambient Temperature zone I think. It's best to confirm.

Good point. It was around +7 F (-14 C) here a few days ago.

We skimmed -30C this morning. :slightly_smiling_face:

Wow, that actually looks pretty awesome. Depending on the price, that might be the next home-lab

this is super nice. Need to price these out. Getting ready to spend some cash on a new lab soon

We're having heat issues with the high end high powered CPU models. Plus, their fans can be quite loud.

What kind of heat issues? Are they getting too hot and then shuts down?

They get too hot to touch, fan runs 110% and loud, and if things are tight enough in the area they reside in they will shut down.
They tend to scale down the CPU before shutdown which is really annoying to the user.

Sounds like something Intel should take a look at otherwise they may not be that good a candidate for a home lab

I don't get the point of putting the NUC bits into a Mini-ITX setup. Why not run with a Mini-ITX board that supports Xeon if that's the way one needs to go?
Most boards we work with support Bifurcation, meaning splitting up that 16x PCIe slot into four (4) PCIe 4x slots. We can then plug in a daughter card that supports 4x M.2 drives to get the bandwidth/IOPS/storage volume we need.

Asus HYPER (we have a few of these)
Tyan P2111-2M + M2110-2M (we have a bunch of these)

Plus, one can use standard sized DIMMs. SO-DIMMs are extremely expensive in their 16GB form.

Nevermind, I found it :slightly_smiling_face:

Through Canadian distribution I think we were something like $80 for the Tyan Pair.

As an FYI: For any lab scenario I suggest only running with SSD/NVMe drives that have power loss protection included. Cost wise, they will be a bit more but all it takes is one power event or blip to kill an entire lab running on those consumer grade SSDs.

A good server grade UPS is also an important item to have. They will have the necessary low let-through numbers that consumer grade UPS systems don't have plus sine wave power.

<https://store.ui.com/collections/early-access/products/usw-lite-16-poe-beta>

Those slats on all those units are plastic and not very sturdy. At least your contractor agreed to replace the whole unit instead of guessing what pieces might be affected.

Given how sensitive to droppage the units are I think that was the best decision.

Minisplits are truly great. You see a lot back in the eastern US since there would be no easy way to redo duct work and it's more efficient and economical to add one or more of these split units. Glad it's worked out well for you!

How much did the mini-split system for your lab cost?  I'm looking to get something similar for mine, and I need to justify it with my CFO.

My cost may not be typical. My breaker box was maxed out so there was a lot of electrical work involved. The drainage was another big item. They had to go out of my office, across the north side of the attic to get to the house wall in the garage, through the crawl space to the back northeast side of the house where they put the external unit. Then they plugged up the useless a/c vents and the large return vent. 

All that work plus the mini-split was $3,800. They spent seven hours working on the electrical and another six hours working on the drainage. We are in an HOA subdivision and a city with strict building codes. No building permits were required, but whoever buys this house we don’t want to pay to fix any code issues. 

This is a Trane/Mitsubishi 12,000 BTU unit that is quieter than a whisper. 

That's a really good price. You did well!