I spent my day waiting for a system crash.<\/p>\n\n\n\n
We’ve got this serious, hardcore system lockup that happens sometimes<\/strong> with the customer of the moment. Naturally, it only happens when the system is flat-out running their most important computation. Naturally, it can reasonably said that Basically, if I get all the systems rocking on a parallel task, then sometimes<\/em> after a few hours, one of them will crash hard enough that it doesn’t respond to So, I started down the list: Power, cooling, bad memory, flaky filesystems, …<\/p>\n\n\n\n At the same time I was working down another list: Too much memory use? Too many tasks, colliding on some secret lock-file? Oversubscribing the NFS server<\/p>\n\n\n\n And yet a third: Bad input? Crappy data files?<\/p>\n\n\n\n Finally, I would get the system rocking and go on to other tasks … until after a while I would exclaim:<\/p>\n\n\n\n MOTHER-FUCKER!<\/p>\n\n\n\n Then I would try again.<\/p>\n\n\n\n Thought I got it. I really did. Left. Dropped the laptop at the hotel. Went to yoga. Good, relaxing, and the knee even handled it well.<\/p>\n\n\n\n When I got back to the hotel, I looked at the computer and said:<\/p>\n\n\n\n MOTHER-FUCKER!<\/p>\n\n\n\n Debug debug debug …<\/p>\n","protected":false},"excerpt":{"rendered":" I spent my day waiting for a system crash. We’ve got this serious, hardcore system lockup that happens sometimes with the customer of the moment. Naturally, it only happens when the system is flat-out running their most important computation. Naturally, it can reasonably said that…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26,33],"tags":[],"class_list":["post-2109","post","type-post","status-publish","format-standard","hentry","category-consulting","category-technology"],"_links":{"self":[{"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/posts\/2109","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/comments?post=2109"}],"version-history":[{"count":1,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/posts\/2109\/revisions"}],"predecessor-version":[{"id":2110,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/posts\/2109\/revisions\/2110"}],"wp:attachment":[{"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/media?parent=2109"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/categories?post=2109"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dwan.org\/index.php\/wp-json\/wp\/v2\/tags?post=2109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}if this code doesn't work, then the system is useless to us<\/code>.<\/p>\n\n\n\nping<\/code>. Naturally this takes the whole parallel job with it. Ctrl-alt-delete (through the KVM) doesn’t cut it. Needs a finger-on-the-button hard reboot. Of course, I have remote control over the power outlets for these systems … do I don’t have to fly to Maryland every time this happens … but still.<\/p>\n\n\n\n