How is unix special 1

[Netwrkengeer]

I'm a Win/Mac guy.
I want to learn Unix, but I need to understand some of its abilities first, some of the things that make it special in a business environment, why do corporations choose Unix over other OS's and what do they use it for. I'd like to emphasize the, what are they using it for part.

Thanks

 

[rycamor]

See posts by myself and others about this topic: thread80-68137

OK, I had some time tonight, so you have to wade through a whole mess of my musings ;-). There is very little Unix can do that other operating systems can't. The big difference is in HOW they do it. Picture your childhood toys as a metaphor:

1) G.I. Joe Battle Copter

2) Lego

With the G.I. Joe helicopter, or the Cobra airplane, you could do some cool stuff, press some buttons, make sounds, make missiles fire, hit the eject button or what-have-you, but eventually, one of two things would happen first: either the toy would break, or you would get bored with it. But... you could always go back to the toy that you never really could get bored with, because you could always build something else out of it (lego). Now the only way to add to the GI Joe stuff was to buy more GI Joe accessories, but each one of those was also limited. With lego, you could add to your collection piece by piece, and keep on rebuilding things the way you wanted. Windows is the GI Joe toy set, while Unix is the lego set.

The completely modular nature of Unix makes it quite a different animal from most other OS approaches. What one person does with Unix could be completely different than another one. I have a version of Linux that fits on a floppy, but contains a perfectly useable command line terminal, and at least 120 of what are considered the core utilities of Unix. I also have the FreeBSD CD set, which contains 4 CDs, including not only the complete FreeBSD core OS, but just about popular Unix program ever written.

So you see one of the main advantages of Unix is its flexibility. Also, like Lego, you can concentrate on one small piece at a time, without really worrying about how they connect to everything else, because that can always be done later. You can write small scripts to automate your most commonly done tasks, then you can write a bigger program that talks to any of these small scripts. You can choose to send the output of a script to the screen, the printer, a text file, someone else's screen, to a database, you name it. The concept of STDIN (standard input) and STDOUT (standard output) is amazing. It essentially means anything can talk to anything. (also called piping). Windows implements some of this functionality from the command line (MSDOS prompt), but it is so limited its practically useless.

Yes, Unix is more difficult to learn. I won't even pretend you will really feel comfortable without spending at least a year using it. But once you start realizing that with this OS, YOU are the one in control, you will get a feeling of power that you just don't get with Windows.

What are corporations using Unix for? Just about anything where data is centralized, stored, served out, etc... Especially where complex operations need to be performed on the data. Perl, which is a scripting language closely tied with Unix, is king of complex operations on data. Apparently most of the serious programmers on Wall Street use Perl on Unix systems to manipulate financial data. When Tim O'Reilly (

http://www.oreilly.com)

started doing surveys of this, he found that many of them were afraid to admit they used Perl, because they considered it a competitive advantage. He had to burst their bubble and tell them they ALL used it.

Traditionally Unix has been stuck in the back room where the Serious data operations are performed, while the departmental networks were run from Windows NT servers. However, more and more companies are finding that they have an easier time with Unix servers, even for their small departmental needs.

Case in point: I had a client with a small insurance business (4 people) who bought a networkable quoting and management system. Rather than spending extra money on a new computer and Windows NT Workstation, I just found a simple Pentium 90 with 16 MB RAM and two 450 MB hard drives. I installed Linux on it, with the Samba software to let it emulate a Windows workgroup server, and it ran for a year and a half without needing a reboot. They loved it. I finally powered it down, just because I wanted to install new hard drives and RAM, upgrade the BIOS, etc...

 

[KenCunningham]

Sir/Madam, an excellent summary of the advantages of unix as an operating system, if I may say so. The only quibble I have is that GI Joe was re-christened 'Action Man' in the UK!!

 

[rycamor]

Well, I suppose the term G.I. didn't mean anything in the UK. It comes from an Americanism based on the phrase "Government Issue", and I think was originally used by 1940s gangsters (somewhat respectfully) to describe the FBI men who hunted them down, but later was used to describe any tough fighting man hired by the government.

Like any typical Unix guy, I am full of useless bits of information. We don't carry over the minimalist philosophy into our own intellectual lives. <chortle> (I will now file away the 'Action Man' diff for later use.)

 

[KenCunningham]

Excellent, always pleased to add to the sum of human knowledge ;o)

Actually, I knew what GI Joe was from a misspent childhood reading Marvel comics and chewing Bazooka Joe gum! Cheers.

 

[CitrixEngineer]

Brilliant analogy, rcycamor.

One minor point;

I think most flavours of UNIX are closer to Meccano than Lego.

Lego = cheap, plastic toy with functionality very close to the real thing (especially in the case of Lego Technic or then new Robot stuff). Breaks apart fairly easily, but has plenty of fancy gimmicks.

Meccano = Expensive, old kit for making robust, well bolted-together products with all the gears and cogs you could wish for. It's not so clear how you put the components together, but when you do it's pretty solid and unbreakable.

Maybe Lego = Linux?


Just a thought

 

[Netwrkengeer]

Thanks,

Now let me ask you this. What do you feel are the necessary skills that a Corporate UNIX Server Admin, should have. If I was to go into an interview and say &quot;Hi I know Unix&quot; what kind of questions would they ask me, and what answers should I have.

 

[MikeLacey]

rcycamor and CitrixEngineer -- I like both of those.

AiX = Meccano, Linux = Lego

yep... Mike
michael.j.lacey@ntlworld.com
Email welcome if you're in a hurry or something -- but post in tek-tips as well please, and I will post my reply here as well.

 

[rycamor]

Having played with Mekkano also, I will have to agree. And FreeBSD tends more toward the Mekkano side.

So what OS matches Lincoln Logs? (stable but boring...)

 

[rycamor]

Netwrkengeer,

There is such a wide range of skills that can be associated with Unix, that its hard to say what qualifies as a 'baseline' level of competence, but I'll take a small stab (others may fill in areas I've missed)

1. Familiarity with the command line. Ability to find one's way around the systems quickly in either the Bash shell, the Korn shell, or a few other of the most common command-line interfaces to Unix. You can actually move around in a system faster this way than with a Windows-style GUI, especially when you get used to some of the advanced features, such as autocompletion of directory and filenames, pushd and popd, etc...

2. Ability to handle at least one good command-line text editor, such as vi, emacs, (or their 'lesser' cousins such as pico, ee, etc...)

3. Shell scripting proficiency: can you write scripts to automate repeated tasks. Can these shell scripts be expanded to receive command-line arguments. (this is how you start building your own toolkit of reusable pieces)

4. Knowledge of source code compiling operations for standard Unix software, so you can take a compressed .tar.gz file, uncompress it, read the compile notes, edit the Makefile, ./configure it, make, make install, etc...

5. Ability to explain each configuration file in /etc (at least most of them)

6. General knowledge of the system layout, such as where are the */bin */lib, */include, */etc type directories and what goes in them.

7. Should be able to change basic networking configuration (IP address, subnet, hostname, domain etc...) in less than a minute. I know some people who can do it in seconds.

8. Ability to use troubleshooting tools such as ifconfig, netstat, tcpdump, etc...

9. Disk formatting and mounting operations (can you add a disk to your system and make it take over the /home directory, for example.)

10. Security...security...security: Do you know the most common exploits of your system? Do you know how to take a default install of whatever system you have, and nail it down, so that there are no unnecessary ports open, and all known vulnerabilities are patched? (a small note... if you use telnet instead of SSH, that immediately puts you in a lower class of Unix user)

The above are all very general. If you mention a specific area of Unix use, such as webserving, or databases, then with each of these should come another list of baseline proficiencies.

 

[Netwrkengeer]

Rycamor,
DataBases, (file server, Print server, App server, etc.) Mostly Hp-UX and Solaris.

On another note, you have been very thorough and helpful. But now I have 10x the questions. But I will ask them in short burst, after you fill me in on the Databases list of baseline proficiencies.

PS If you feel like I'm killing you with questions, just let me know.

 

[rycamor]

>>PS If you feel like I'm killing you with questions, just let me know.

Hehe... Sometimes questions like these are good, because they stimulate me to remember the areas I've promised myself to study, but keep putting off. Obviously, you have some familiarity with Unix, as evidenced with post thread80-70838 (So are you just asking these questions rhetorically, to provoke discussion...?)

First off, I have been using Unix for 3 or 4 years, and before that I was much more familiar with Windows. This gives me a position of strength when it comes to explaining things to Windows/Mac people, but I am not the seasoned expert that some in this forum are, such as Mike Lacey.

I have not had the chance to play with &quot;big iron&quot; Unix very much (a little Solaris), but mainly my experience comes from Linux, and then my favorite: FreeBSD (

http://www.freebsd.org).

The good thing about FreeBSD is that it comes from the family tree of the grandaddy of all Unixes, while Linux is really just a (very good) carbon copy.

I am a web application developer, spending most of my time on Apache, PHP, Perl, etc... with of course the obligatory database involvement, but I have not worked on any serious data warehousing, ERP, GIS, or some such. My experience with Unix databases comes mainly from MySQL, some PostgreSQL, and one Sybase seminar . Theoretically, databases should not vary much from OS to OS, but a few things that stand out with Unix databases are:

1) Command line operation all the way. Generally, with a Unix database, you accomplish everything either form the command line or programmatically, unlike MSSQL, in which you are expected to work in the GUI. Generally each of these has a specific SQL command-line environment, and each of these is a little different, so with each one, there is a bit of orientation. Again, there is great flexibility with STDIN and STDOUT in being able to talk to this SQL client. For example, in MySQL, I can run a command line call to the client, without actually entering _into_ it, in order to pipe in a query from a text file (

#mysql -u username -p < myquery.sql

) or run a dump command to output the contents of a database into a text file (

 mysqldump -u username -p databasename > myoutput.sql

). From there, you can do more complex things such as piping in and out at the same time, etc... This leads to:

2) Maintenance, backup, etc... Most Unix-based databases, expect you to decide when you want to optimize tables, etc... so it is up to you to write your own scripts that handle periodic backups, optimizations (like defragmenting), etc... You should be familiar enough with your system that you can dump a database to a text file (as a series of SQL commands), edit it to fix problems in structure or whatever, and then turn it back into a database, rename tables, etc... and you should know where the binary files that contain the actual data are, and what you can and cannot do with those. Does you database support replication, hot copies, etc..., you need to know these things, and know how to take advantage of them.

3) Performance and scaleability: The more serious DBMs, such as SyBase, often allow you do to such things as mount a disk as a raw device specifically under the database's own filesystem, so it is optimized just for that purpose. If you mount several disks as SyBase devices, it will automatically make them into a software RAID, serving only your database. Beyond this, there are many tweaks you can do, especially at compile time, to optimize a database for your application. MySQL, for example, defaults to supporting many international character sets, but if you only need your language, you can compile just for one character set, thus having a much leaner database. Databases can also be scaled by tieing them together through a network into multiple machines, so you can split up your tables, and have different tables on different machines, while the client application sits on yet another machine and simply sends queries and gets responses. There are many more sophisticated things that can be accomplished besides that. For example, with MySQL, there is a Clustering Toolkit (

http://prometheus.zerodivide.net/mysql_cluster/),

that experiments with schema for making a database share its data between multiple servers, as if each were a disk on a RAID system. There are many systems like this in the Serious database world (which I do not have experience with)

4) Then of course there is all the programming stuff, how you actually deal with the database, which is too involved to go into here, but anyone dealing with databases should know ANSI SQL, should know what table or row locking is all about, what transactions are, indexes, keys, Foreign Key Constraints, stored procedures, etc... and which of these your database supports, and how, and maybe even why. (For MySQL, even thought its considered a 'lightweight' database, to the credit of the developers, they explain in detail the how and why of all design choices on the site), as does PostgreSQL, also an open source, but more serious database. (Actually PostgreSQL claims to have &quot;better than row level locking: object level locking&quot. Lastly but not leastly, you should have a clue what is meant by &quot;normalization&quot;, or your database is doomed to failure.

OK, I've said enough. Others can fill you in much better than I on file serving, print serving, app serving, the finer points of database systems, etc...

If you have any questions about webserving, meet me in the Apache forum. (Let's not pretend Unix can involve another webserver besides Apache ;-) )

 

[Netwrkengeer]

>>Obviously, you have some familiarity with Unix.
yes, very little and mostly linux.

>>So are you just asking these questions rhetorically, to provoke discussion.
Yes and no. Here is the deal, I don't know unix enough to give an answer like the one I posted in Thread 80-70838 as you stated. But I do know unix enough to know where and what to look for when I need answers. Hence the post in thread 80-70838.

And I'll see you in the apache forum.

 

[Netwrkengeer]

Rycamor:

You said you have experience with many DB's for the web, If I was to build a site(business) on the net that received over a million hits a week (&quot;i'd be very rich&quot what back end database for holding user info would you suggest.

I know oracle is the best, but how about a DB you can DL for free.

Thanx

 

[rycamor]

The two best (at least best-and-most-popular) are MySQL and PostgreSQL, each of which has a forum here. Mainly my experience is with MySQL, but I have played around with PostgreSQL now and then. You will have the easiest time if you use Apache, PHP, and MySQL, as integration between these three is now as seamless (and more) as any integrated web suite out there (IIS/ASP/MSSQL, Websphere, etc...) There is actually a website by O'Reilly called OnLAMP (Linux/Apache/MySQL/PHP) at

http://www.onlamp.com/.

MySQL (

http://www.mysql.com)

has become the most popular web-connected database system for open-source web software. This is because:
1) it is easy to install and use
2) it is FAST
3) it is rock-solid. I have never yet had a MySQL database corrupt. I know it does happen, but very rarely.

It doesn't have all the functionality of the most Serious databases (yet), but for web-based apps those are not always used. What MySQL doesnt have:

1) stored procedures
2) sub-selects
3) foreign-key constraints
4) row-level locking (that is supposed to happen soon)
5) Transactions are not MySQL's strong suite -- it DOES have transactions in the version, but they are brought in as 3rd party libraries (Berkely DB tables), and I have no experience with them. The cool thing is, you can mix a database with transaction-aware and non-transaction-aware tables, so you can optimize the ones that don't need transactions.

For more serious data processing needs, PostgreSQL has everything that MySQL doesn't have, including a few things that not even the major players have, such as something they call &quot;better than row-level locking; object-level locking&quot;, transactions, stored procedures, user-defined datatypes, etc...

PostgreSQL plays well with PHP or Perl, but is not yet as smoothly integrated with PHP as MySQL (another situation that is being dealt with)

MySQL has the fastest performance for fast concurrent reads (SELECT operations), but gets a little bogged down with mixed inserts, updates, selects, etc...

PostgreSQL (

http://www.postgresql.org)

has sort of an opposite performance ratio to MySQL, as befits it's concentration on serious computing needs, so its performance is good for many different types of operations happening concurrently.

The long and short of it is: if you need a website which serves dynamic content where you might have a million reads from the database for every page worth of data you input, then use MySQL. If you have a serious e-commerce or banking website where you can't afford to have non-atomic operations, or a web-based app that involves intensive, complex updates from many different users, then use PostgreSQL.

Or better yet, use them both at the save time, each for the operations that suit it best. Each of these databases, it seems, started from different ends of the spectrum and is evolving toward the other, so in the end the open source world has two very strong contenders for database management.

 

[Netwrkengeer]

Rycamor
Perfect answer. everything you stated is identical to the research I have done on the 2 DBs.

Now for a selfish question, I need to setup a DB to interact with mapping software, I'm using MySql on RH Linux 6 with Php, I need it to query and post like mapquest and also query and post internally to feed itself data, For example, John wants to know where the closest super market is, the DB has johns Lat,Long data, and the lat,long data of 15 super markets, we must match him up with the closest one. SQL will query first to get the data when John signs up. the post internally, same with the super markets. Second john runs a search, the DB needs to query and post externally for john to go get dinner.

Now the question, do you know of any mapping software (preferably free) that I could have interact with my DB?

And do you have any tips on how to set this up?

 

[rycamor]

I don't know of any full-featured mapping software that compares to mapquest. Such a job is quite production-intensive, and I doubt there is anything like it in the open-source camp.

If you are just looking for a method of comparing latitude and longitude to find closest matches, and return the exact distance, or even the nearest 10 closest matches, that's a simple matter of a couple of formulas (which I happen to have --they were easy enough to find on the &quot;Ask Dr. Math&quot; forum:

http://forum.wharthmore.edu/dr.math/).

If you need to actually *display* a map, your best bet would be to just send the user to the appropriate Mapquest query. I believe you can even work out an agreement with Mapquest to query their system and post the map back in your dynamic webpage.

 

[Netwrkengeer]

Thanx
what was that forum, your link doesn't work

 

[Netwrkengeer]

Rycamor,
I found dr.math on

http://mathforum.com/dr.math/

and
I have another question

Can RH Linux6 mirror IDE drives, and how do you do it.

Can it be done on the same channel?
Can it be done on the different channels?