Notes

OpenBSd or HP-UX? interesting discussion on (Open)LDAP on OpenBSD or HP-UX, on a large or a set of smaller machines, etc.

marc.info - openbsd-misc - Compilers in OpenBSD
marc.info - openbsd-misc - Intel Core 2 talks about processor bugs.

Date: Thu, 8 Jan 2009 16:32:41 -0800
From: "Gerald Chudyk"
To: "Peter Kay - Syllopsium"
Subject: Re: Best supported arch/workstation
Cc: "Matt KP60", misc.openbsd.org

> I've never personally had any issues with the sparc port (32 bit), but all
> those machines are old and slow.
>
Hush, my little inetra. I'm sure he wasn't really talking about you.
You are very good at what you do. And you're right: it's not your
fault I haven't found a replacement for your dead nvram battery.

Once I remind you about who you are, and what you should be doing,
you never forget anything until the next unplanned power event.

And besides it's not how much processor power or memory you have or
how big your hard drive that makes you attractive to me. It's the
little things we have shared together over time: dns, dhcp, sendmail,
apache. I could go on and on.

Whew...that was a close one.

Subject: 	Re: nfsv4?
Date: 	Wed, 27 Oct 2010 19:34:15 -0600
From: 	Theo de Raadt
To: 	<...>
CC: 	<...>, misc.openbsd.org
<snip>
shit which comes out of research organizations all tends to suck these
days, doesn't it.  or perhaps it always did (OSI networking, ipv6,
same same).

i have theorized in the past that the problem we face is
that an insufficient number of axe murderers are attending those kinds
of research meetings.

Subject: 	Re: I don't get where the load comes from
Date: 	Wed, 01 Jun 2011 15:41:51 +0200
From: 	Benny Lofgren 


On 2011-06-01 15.12, Joel Wiramu Pauling wrote:
> Load is generally a measure of a single processor core utilization over an
> kernel dependent time range.

No it isn't. You have totally misunderstood what the load average is.

> Generally as others have pointed out being a very broad (not as in meadow,
> as in continent). Different OS's report load very differently from each
> other today.

That one's sort of correct, although I've yet to see an OS where the load
doesn't in some way refer to an *average* *count* *of* *processes*.

> Traditionally you would see a load average of 1-2 on a multicore system (I
> am talking HP-UX X client servers etc of the early 90's vintage). a Load
> average of 1 means a single core of the system is being utilized close to
> 100% of the time.

No, no, no. Absolutely *NOT*. It doesn't reflect CPU usage at all.

And it never have. The load average must be the single most misunderstood
kernel metric there have ever been in the history of unix systems.

Very simplified it reflects the *number* *of* *processes* in a runnable
state,
averaged over some time. Not necessarily processes actually on core,
mind you,
but the number of processes *wanting* to run.

Now, a process can be in a runnable state for a variety of reasons, and
there
is for example nothing that says it even needs to use up its alloted time
slice when actually running, but it still counts as runnable. It can be
runnable when waiting for a system resource; then it consumes *no* CPU
cycles
at all, but it still counts towards the load average.

> On dual core systems a load average of 1 should be absolutely no cause for
> concern.

I routinely see load averages of 30-40-50, upwards of 100 on some of my
systems. They run absolutely smooth and beautiful, with no noticable lag
or delays. The processors may be near idling, they may be doing some work,
it varies, but it is nothing I can tell from the load average alone.

> Linux has moved away from reporting load average as a percentage of a single
> core time in recent days for precisely this reason, people see a load of 1
> and think there systems are esploding.
> 
> In the traditional mold todays processors should in theory get loads of 4-7
> and still be responsive...

I'm sorry to say, but your entire text is based on a misunderstanding of
what
the load average really is, so the above sentences are totally irrelevant.


Regards,
/Benny


> On 31 May 2011 19:10, Joel Carnat <...> wrote:
> 
>> Le 31 mai 2011 ` 08:10, Tony Abernethy a icrit :
>>> Joel Carnat wrote
>>>> well, compared to my previous box, running NetBSD/xen, the same services
>>>> and showing about 0.3-0.6 of load ; I thought a load of 1.21 was quite
>> much.
>>>
>>> Different systems will agree on the spelling of the word load.
>>> That is about as much agreement as you can expect.
>>> Does the 0.3-0.6 really mean 30-60 percent loaded?
>>
>> As far as I understood the counters on my previous nbsd box, 0.3 meant that
>> the
>> cpu was used at 30% of it's total capacity. Then, looking at the sys/user
>> counters,
>> I'd see what kind of things the system was doing.
>>
>>> 1.21 tasks seems kinda low for a multi-tasking system.
>>
>> ok :)
> 

Subject: 	Re: I don't get where the load comes from
Date: 	Wed, 01 Jun 2011 17:26:37 +0200
From: 	Benny Lofgren


On 2011-06-01 15.53, Joel Wiramu Pauling wrote:
> On 2 June 2011 01:41, Benny Lofgren <...
> <mailto:...>> wrote:
> I agree with what you are saying, and I worded this quite badly, the
> frame I was trying to setup was "back in the day" when multi-user meant
> something (VAX/PDP) - the load average WAS tied to core utilization - as
> you would queue a job, and it would go into the queue and there would be
> lots of stuff in the queue and the load average would bumo, because
> there wasn't much core to go around.

Not wanting to turn this into a pissing contest, I still have to say that
you are fundamentally wrong about this. I'm sorry, but what you are saying
simply is not correct.

I've worked in-depth on just about every unixlike architecture there is
since I started out in this business back in 1983, and on every single
one (that employed it at all) the load average concept has worked
similarly to how I described it in my previous mail. (Not always EXACTLY
alike, but the general principle have always been the same.)

The reason I'm so adamant about this is that the interpretation of the
load average metric truly is one of the longest-standing misconceptions
about the finer points of unix system administration there is, and if
this discussion thread can set just one individual straight about it
then it is worth the extra mail bandwidth. :-)

One only needs to look at all of the very confident, yet dead-wrong,
answers to the OP:s question in this thread to realize that it is
indeed a confusing subject. And the importance of getting it straightened
out cannot be overstated. I've long ago lost count of the number of
times I've been called in to "fix" a problem with high system loads
only to find that the only metric used to determine that is... yes,
the load average. I wonder how much money have been wasted over the
years trying to throw hardware on what might not even have been a
problem in the first place...


Regards,
/Benny



> That hasn't been the case for a very very long time and once we entered
> the age of multi-tasking load become unintuitive.
> 
> Point being it's an indication of something today that isn't at all
> intuitive.
> 
> Sorry for muddying the waters even more, my fuck up.
> 
> 
>     > On 31 May 2011 19:10, Joel Carnat <...
>     <mailto:j...>> wrote:
>     >
>     >> Le 31 mai 2011 ` 08:10, Tony Abernethy a icrit :
>     >>> Joel Carnat wrote
>     >>>> well, compared to my previous box, running NetBSD/xen, the same
>     services
>     >>>> and showing about 0.3-0.6 of load ; I thought a load of 1.21 was
>     quite
>     >> much.
>     >>>
>     >>> Different systems will agree on the spelling of the word load.
>     >>> That is about as much agreement as you can expect.
>     >>> Does the 0.3-0.6 really mean 30-60 percent loaded?
>     >>
>     >> As far as I understood the counters on my previous nbsd box, 0.3
>     meant that
>     >> the
>     >> cpu was used at 30% of it's total capacity. Then, looking at the
>     sys/user
>     >> counters,
>     >> I'd see what kind of things the system was doing.
>     >>
>     >>> 1.21 tasks seems kinda low for a multi-tasking system.
>     >>
>     >> ok :)
>     >

Subject: 	Re: I don't get where the load comes from
Date: 	Wed, 01 Jun 2011 09:49:17 -0600
From: 	Theo de Raadt
To: 	Benny Lofgren


> On 2011-06-01 15.53, Joel Wiramu Pauling wrote:
> > On 2 June 2011 01:41, Benny Lofgren <...
> > <mailto:...>> wrote:
> > I agree with what you are saying, and I worded this quite badly, the
> > frame I was trying to setup was "back in the day" when multi-user meant
> > something (VAX/PDP) - the load average WAS tied to core utilization - as
> > you would queue a job, and it would go into the queue and there would be
> > lots of stuff in the queue and the load average would bumo, because
> > there wasn't much core to go around.
> 
> Not wanting to turn this into a pissing contest, I still have to say that
> you are fundamentally wrong about this. I'm sorry, but what you are saying
> simply is not correct.
> 
> I've worked in-depth on just about every unixlike architecture there is
> since I started out in this business back in 1983, and on every single
> one (that employed it at all) the load average concept has worked
> similarly to how I described it in my previous mail. (Not always EXACTLY
> alike, but the general principle have always been the same.)
> 
> The reason I'm so adamant about this is that the interpretation of the
> load average metric truly is one of the longest-standing misconceptions
> about the finer points of unix system administration there is, and if
> this discussion thread can set just one individual straight about it
> then it is worth the extra mail bandwidth. :-)

100% right.  The load average calculation has not changed in 25 years.
Anyone who says otherwise hasn't got a single fact on their side.

What has changed, however, is that the kernel has more kernel threads
running (for instance, ps aguxk, and look at the first few which have
the 'K' flag set in the 'STAT' field.

Some kernels have decided to not count those threads, others do count
them.  Since these kernel threads make various decisions for when to
do their next tasks and how to context switch, the statistical
monitoring of the system which ends up creating load values can get
perturbed.

That's what this comes down to.

-------- Original Message --------
Subject: 	Re: openbsd clusters
Date: 	Sat, 22 Dec 2012 22:43:54 -0500
From: 	Nick Holland


On 12/22/12 07:54, Friedrich Locke wrote:
...
> But for other services i don't have now what i could use. A example: i need
> a file system that must expand by adding more machine in the network in a
> simple way.

in plain English: "I'm not thinking out the design carefully, so I'm
going to rely on fancy shit to haul my ass out of the fire when the
predictable (and not so predictable) happens.

You don't need that for your problem, you need that for the solution you
came up with for your problem.  Your solution is wrong.

You know your needs will change in the future, so build the whole system
around the idea of modular storage and other scalability design features
-- not "unlimited expandable storage".

Chunk your data from the very beginning.  In the case of a mail server,
part of the user's LDAP record indicates the storage unit where it is
stored.

Yes, this is a better design.

I've seen many designs where the answer was "toss it all in one pool,
let some 'advanced technology' keep my ass out of the fire."  They have
all been total shit.  Usual result: the "advanced technology" gathers
the kindling, splits the logs, lights the fire, and tosses your ass on
the pyre before you ever get around to the first "expansion".  If you
wish to argue that your "problem" is special, and requires One Big Pool
of Storage, feel free to tell me about it (off list), maybe someone's
got one.  More likely, you will be telling me about your SOLUTION which
requires one big pool, not the root problem.  (I'm not above learning
new stuff, but I'm done with assuming most people know something I don't
-- that's something that is really annoying to be wrong about, I'm finding).

Your design should incorporate (among other things):
* initial load handling.
* future load handling improvements.
* future storage upgrade.
* future storage REPLACEMENTS (you want to remove your three year old
storage module in favor of a new one ten times the size, but your six
month old one is still quite good)
* future complete solution replacements. (*)
the simplest possible solutions that will accomplish the above within
acceptable business frameworks (i.e., not "we'll have our entire IT
staff working a major multi-day holiday because that's the only way we
can accomplish this")

Nick.


(*) if you ever wish to keep a closed source solution OUT of your
operations, this is your magic weapon to use with responsible, thinking
people.  Every closed source solution is built around the idea of
keeping you a captive customer.  But the fact is, if your business is
run well, in 50 years, it can still be around.  You will almost
certainly have to replace entire systems with competing products "some
day" -- your company's success should not be dependent upon a third
party remaining in business.  So, an exit strategy has to be part of any
good system design (even though it almost never is).  How are you going
to scrape your legacy data off your old system and install it into its
replacement?  When the APIs are proprietary, you won't...  Ask your
prospective vendor "If you go bankrupt or otherwise leave the business
next year, how will we move >OUR< data stored in your system to another
product?"  They will start with "We aren't going anywhere", which you
know they would say if they weren't sure about getting their paychecks
next week.

'course, most people are not thinking about the long-term health of the
company, but the short-term "what can I stuff on my resume on my way out
the door before this blows up"