[BLUG] NEXT BLUG meeting- July 15 at the library

All-

I'm pushing the meeting back by one week this month, to July 15th (a
week after we'd normally have it) and have updated the wiki to reflect
this (yep, I remembered to update the wiki; shocking, I know!).

What: Next BLUG Meeting
Why: Mark Krenz will be demonstrating Asterisk
Where: Monroe County Public Library, meeting room 1A
When: Tuesday, July 15 2008

I'll be very difficult to reach from the 4th (Friday) up until a day or
two before the meeting; if you write and I don't respond, don't be
surprised.

What is Asterisk? Asterisk is an open source phone system for Linux.
It can run (with special hardware) connected to POTS (plain old
telephone service; the outside world), or it can run as a pure VOIP
system using IP phones and "softphones" (with your mic and headset, or
similar). It can be connected to a single line or many lines and then
run to extensions for everybody with voice mail, on-hold music, etc (if
you like).

Asterisk is mature, *very* full-featured, relatively easy to program (as
they say: easy to learn, a challenge to master), and suitable for
anything from talking amongst friends, to an enterprise phone system for
your company (I really mean that).

If you have any interest in telephony, VOIP, or real-time communication
via the internet, I strongly suggest you come along and check out Mark's
talk.

Happy Independence Day!
Cooley

--
Dave Cooley
dcooley@kiva.net

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

It doesn't look like you were using tar??? If you use tar, you should
have only one file and you don't need 'scp -r'; e.g.,

cd /home
tar cvzf user.tar.gz user
scp user.tar.gz remote_machine:

Later you do

cd /home
scp remote_machine:usr.tar.gz
tar xvzf usr.tar.gz

Am I missing something?

Shing-Shong

>
> Here's one more. Before any fresh install, I used to do tar -xvzf
> on /home/user, then scp -r the tarball to another machine, then scp -r
> back after the install. But all sorts of oddments got messed up; I
> found out eventually that tar -xvzf doesn't follow symlinks.
>
> There's another switch for that, I disremember which. Instead of
> learning it, I got lazy, and just took to using some third machine
> while scp took up the bandwidth -- this on a mere home LAN, of course.
>

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, 2 Jul 2008, Brian Wheeler wrote:

> ... Gnu tar does a pretty good job of getting it right, but
> there are still surprises lurking for the unwary.

Here's one more. Before any fresh install, I used to do
tar -xvzf on /home/user, then scp -r the tarball to another
machine, then scp -r back after the install. But all sorts of
oddments got messed up; I found out eventually that tar -xvzf
doesn't follow symlinks.

There's another switch for that, I disremember which.
Instead of learning it, I got lazy, and just took to using some
third machine while scp took up the bandwidth -- this on a mere
home LAN, of course.

--
Beartooth Staffwright, Not Quite Clueless Power User
Fedora 8 & 9; Alpine 1.10, Pan 0.132; Privoxy 3.0.6;
Dillo 0.8, Epiphany 2, Firefox 2 & 3, Galeon 2, Opera 9
Remember I know little (precious little!) of where up is.

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

I have to correct myself since my explanation was wrong.

> scp -p files root@machineB:

In this case, you log into machine B as root and then create the 'files.'
Thus it will be owned by root. But if you do

scp -p files user@machineB:

it login as 'user' and create the file; thus owned by user (and the file
is right under user's home directory.)

Shing-Shong

> This is because the files on machine A is created by root, the user who
> is running the scp. If you want to preserve all information, you need
> to use tar:
>
> ssh machineB -n tar cCf /path/to/file - . | tar xf -
>
> (Or you have to 'su X' on machine A before you do the scp.)
>
> Shing-Shong
>
> Mark Krenz wrote:
>> I have some files that are owned on machine A by user X. As the root
>> user on machine A, I run
>>
>> scp -p files root@machineB
>>
>> The mode and modification times are preserved, but the ownership is
>> not. I know that with rsync -a the ownership is preserved, but
>> according ot the scp man page, ownership isn't preserved with -p. I
>> just got burned by this. Tell me if I'm insane or not but I could swear
>> that this worked before. It might have had this in a much older version
>> of scp, like the old non-openssh version of ssh.
>>
>> Know your options.
>>
>>
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

I always use dump/restore to clone machines like this. --SS

>> Ah! I know why the --numeric-uid option made a difference. You were
>> copying the entire system, including passwd/group files. This means
>> that the first time you copied the files, midway through the process
>> passwd/group were over-written and what had earlier been the sendmail
>> user became someone else.
>>
>>
>
> Yep. I was making a physical machine into a virtual machine with a
> different sized disk :)
>
>
>
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, 2008-07-02 at 13:45 -0400, Steven Black wrote:
> On Wed, Jul 02, 2008 at 01:05:01PM -0400, Brian Wheeler wrote:
> > We use the UITS UID on our machines so we get some really large UIDs...I
> > make it a point to always specify the new user uid just so I don't
> > clobber a future employee by accident.
>
> Do you use NFS? I also work at UITS, but don't make use of any remote
> storage, so using a common network UID is just more work. It isn't
> standard (particularly not standard for Debian or Ubuntu) so the
> cost/benefit just isn't there for me.
>

We're not using NFS, but when the policy got set we were using DFS so we
had to keep in sync with UITS. Now its more habit than anything else.

> Now, I've thought about automatically adding users as they appear in the
> proper LDAP group. That may make it easier to use the UITS UID. That's
> below a number of current priorities at this point, though.
>

Rumor has it that the posixUID field in Active Directory will be made
available in the not-too-distant future. Also, I believe there may be a
SOAP interface available.

> At this point, I can basically run one command and the user ID is
> added/removed from the majority of the services I control. Things
> aren't perfect yet, but automation is a passion of mine, so things will
> improve.
>

With our servers I have one machine that has a master copy of
passwd/group/shadow and I have cron jobs on the other machines which scp
it over and merge it with the system accounts on those machines (it also
determines if an account is locked and sets the .ssh directory to mode
000 so public/private keypairs aren't available for the locked user). I
use pam + groups to determine actual access to the machine.

> > Here's what I was doing: I booted a rescue CD and did a full system
> > copy from a remote machine to the one I booted the rescue disk on. The
> > system accounts had different uids (sendmail, in particular) so when I
> > booted the resulting system, sendmail was broken because the uid suid
> > binaries didn't match the sendmail uid in passwd.
> >
> > When I recopied it later I used the --numeric-uid option and the IDs
> > were synced up properly.
>
> Ah! I know why the --numeric-uid option made a difference. You were
> copying the entire system, including passwd/group files. This means
> that the first time you copied the files, midway through the process
> passwd/group were over-written and what had earlier been the sendmail
> user became someone else.
>

Yep. I was making a physical machine into a virtual machine with a
different sized disk :)

> Using --numeric-uid prevents this problem as it ignores the destination
> passwd/group files. (It doesn't need them, as they'll be over-written.)
>
> This is totally the reason that --numeric-uid exists. However, the
> reason it isn't the standard is that unless /etc/passwd is being
> touched, it will only aggravate your trouble.
>
> > I didn't explain it properly, but any time there are differences between
> > source/dest password files there may be issues. Gnu tar does a pretty
> > good job of getting it right, but there are still surprises lurking for
> > the unwary.
>
> I concur completely.
>
> Cheers,
> Steven Black
>
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, Jul 02, 2008 at 01:05:01PM -0400, Brian Wheeler wrote:
> We use the UITS UID on our machines so we get some really large UIDs...I
> make it a point to always specify the new user uid just so I don't
> clobber a future employee by accident.

Do you use NFS? I also work at UITS, but don't make use of any remote
storage, so using a common network UID is just more work. It isn't
standard (particularly not standard for Debian or Ubuntu) so the
cost/benefit just isn't there for me.

Now, I've thought about automatically adding users as they appear in the
proper LDAP group. That may make it easier to use the UITS UID. That's
below a number of current priorities at this point, though.

At this point, I can basically run one command and the user ID is
added/removed from the majority of the services I control. Things
aren't perfect yet, but automation is a passion of mine, so things will
improve.

> Here's what I was doing: I booted a rescue CD and did a full system
> copy from a remote machine to the one I booted the rescue disk on. The
> system accounts had different uids (sendmail, in particular) so when I
> booted the resulting system, sendmail was broken because the uid suid
> binaries didn't match the sendmail uid in passwd.
>
> When I recopied it later I used the --numeric-uid option and the IDs
> were synced up properly.

Ah! I know why the --numeric-uid option made a difference. You were
copying the entire system, including passwd/group files. This means
that the first time you copied the files, midway through the process
passwd/group were over-written and what had earlier been the sendmail
user became someone else.

Using --numeric-uid prevents this problem as it ignores the destination
passwd/group files. (It doesn't need them, as they'll be over-written.)

This is totally the reason that --numeric-uid exists. However, the
reason it isn't the standard is that unless /etc/passwd is being
touched, it will only aggravate your trouble.

> I didn't explain it properly, but any time there are differences between
> source/dest password files there may be issues. Gnu tar does a pretty
> good job of getting it right, but there are still surprises lurking for
> the unwary.

I concur completely.

Cheers,
Steven Black

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, 2008-07-02 at 12:27 -0400, Steven Black wrote:
> On Wed, Jul 02, 2008 at 12:01:05PM -0400, Brian Wheeler wrote:
> > True, but you have to make sure that the passwd/group file is in sync
> > for all of the files or there'll be weirdness unless you use
> > --numeric-owner.
>
> What do you mean "there'll be weirdness unless you use --numeric-owner"?
>
> GNU Tar stores both the text name, and the numeric ID for users and
> groups. It favors the text name if possible and falls back to the
> numeric ID if the name is unavailable.

Yep

> This means you can have three machines each with three users. Each
> of the users could have been added to the machines in a different
> order. (So on MachineA, UserA is 1000, etc.) However, GNU Tar will work
> properly in this case, and the files will not get confused.
>
> This actually makes GNU Tar handy to use if you ever find yourself
> working with a machine where the user or group IDs have gotten totally
> out of hand and you want to clean them up. (For instance, if you're
> using a system where useradd always picks the last user ID, and some
> fool of a prior administrator saw an example giving postfix a user ID of
> 54321 and copied that in as-is.)
>

We use the UITS UID on our machines so we get some really large UIDs...I
make it a point to always specify the new user uid just so I don't
clobber a future employee by accident.

> Of course, if you over-write a passwd file on one machine with a passwd
> file from another machine you're basically screwed. It isn't Tar that
> screws you, it is just that you should never-ever do it. (So you're
> basically screwing yourself. It is easy to forget this and do it without
> thinking -- I know I've done it before.)
>

as have I :)

> Most Linux distributions don't guarantee the IDs of even the system
> users/groups. With the system IDs you may be able to find a tool to help
> you correct them. With the user IDs, you'll basically be forced to chown
> the home directories to who they're expected to be, and just correct any
> other files to a user/group that will allow the user community access
> from there.
>

Here's what I was doing: I booted a rescue CD and did a full system
copy from a remote machine to the one I booted the rescue disk on. The
system accounts had different uids (sendmail, in particular) so when I
booted the resulting system, sendmail was broken because the uid suid
binaries didn't match the sendmail uid in passwd.

When I recopied it later I used the --numeric-uid option and the IDs
were synced up properly.

I didn't explain it properly, but any time there are differences between
source/dest password files there may be issues. Gnu tar does a pretty
good job of getting it right, but there are still surprises lurking for
the unwary.

Brian

> Cheers,
> Steven Black
>
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, Jul 02, 2008 at 12:01:05PM -0400, Brian Wheeler wrote:
> True, but you have to make sure that the passwd/group file is in sync
> for all of the files or there'll be weirdness unless you use
> --numeric-owner.

What do you mean "there'll be weirdness unless you use --numeric-owner"?

GNU Tar stores both the text name, and the numeric ID for users and
groups. It favors the text name if possible and falls back to the
numeric ID if the name is unavailable.

This means you can have three machines each with three users. Each
of the users could have been added to the machines in a different
order. (So on MachineA, UserA is 1000, etc.) However, GNU Tar will work
properly in this case, and the files will not get confused.

This actually makes GNU Tar handy to use if you ever find yourself
working with a machine where the user or group IDs have gotten totally
out of hand and you want to clean them up. (For instance, if you're
using a system where useradd always picks the last user ID, and some
fool of a prior administrator saw an example giving postfix a user ID of
54321 and copied that in as-is.)

Of course, if you over-write a passwd file on one machine with a passwd
file from another machine you're basically screwed. It isn't Tar that
screws you, it is just that you should never-ever do it. (So you're
basically screwing yourself. It is easy to forget this and do it without
thinking -- I know I've done it before.)

Most Linux distributions don't guarantee the IDs of even the system
users/groups. With the system IDs you may be able to find a tool to help
you correct them. With the user IDs, you'll basically be forced to chown
the home directories to who they're expected to be, and just correct any
other files to a user/group that will allow the user community access
from there.

Cheers,
Steven Black

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

I really thought SCP always used the ownership of the user at the target
machine. That is, it always ignored the owner. Then again, on all of my
machines root is explicitly disallowed to log in via SSH.

When copying large numbers of files between machines, I usually use a
tar pipe:

sudo tar cf - SourceDir | ssh user@host 'sudo tar xf - -C /TargetBase'

Or (depending on quantity and available network/server bandwidth):

sudo tar jcf - SourceDir | ssh user@host 'sudo tar jxf - -C /TargetBase'

(The above activity may not work reliably unless sudo on the target host
is configured to not to always ask for the password and the password has
been provided to sudo within the timeout period.)

As far as I know, tar pipes are the standard reliable method of
performing this activity.

More frequently, though, I'm actually just making a backup and storing
it elsewhere (as the current machine doesn't have space to store it):

sudo tar jcf - SourceDir | ssh user@host 'cat > SourceDir.tar.bz2'

Or (depending on source machine's processing power):

sudo tar cf - SourceDir | ssh user@host 'bzip2 -c > SourceDir.tar.bz2'

Cheers,
Steven Black

On Wed, Jul 02, 2008 at 03:37:24PM +0000, Mark Krenz wrote:
>
> I have some files that are owned on machine A by user X. As the root
> user on machine A, I run
>
> scp -p files root@machineB
>
> The mode and modification times are preserved, but the ownership is
> not. I know that with rsync -a the ownership is preserved, but
> according ot the scp man page, ownership isn't preserved with -p. I
> just got burned by this. Tell me if I'm insane or not but I could swear
> that this worked before. It might have had this in a much older version
> of scp, like the old non-openssh version of ssh.
>
> Know your options.
>
> --
> Mark Krenz
> Bloomington Linux Users Group
> http://www.bloomingtonlinux.org/
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

On Wed, 2008-07-02 at 11:57 -0400, Shei, Shing-Shong wrote:
> This is because the files on machine A is created by root, the user who
> is running the scp. If you want to preserve all information, you need
> to use tar:
>
> ssh machineB -n tar cCf /path/to/file - . | tar xf -
>
> (Or you have to 'su X' on machine A before you do the scp.)
>

True, but you have to make sure that the passwd/group file is in sync
for all of the files or there'll be weirdness unless you use
--numeric-owner.

That one bit me when I was using a rescue disk and copying one machine
to another...

Brian

> Shing-Shong
>
> Mark Krenz wrote:
> > I have some files that are owned on machine A by user X. As the root
> > user on machine A, I run
> >
> > scp -p files root@machineB
> >
> > The mode and modification times are preserved, but the ownership is
> > not. I know that with rsync -a the ownership is preserved, but
> > according ot the scp man page, ownership isn't preserved with -p. I
> > just got burned by this. Tell me if I'm insane or not but I could swear
> > that this worked before. It might have had this in a much older version
> > of scp, like the old non-openssh version of ssh.
> >
> > Know your options.
> >
> >
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] scp -p preserving ownership through root user

This is because the files on machine A is created by root, the user who
is running the scp. If you want to preserve all information, you need
to use tar:

ssh machineB -n tar cCf /path/to/file - . | tar xf -

(Or you have to 'su X' on machine A before you do the scp.)

Shing-Shong

Mark Krenz wrote:
> I have some files that are owned on machine A by user X. As the root
> user on machine A, I run
>
> scp -p files root@machineB
>
> The mode and modification times are preserved, but the ownership is
> not. I know that with rsync -a the ownership is preserved, but
> according ot the scp man page, ownership isn't preserved with -p. I
> just got burned by this. Tell me if I'm insane or not but I could swear
> that this worked before. It might have had this in a much older version
> of scp, like the old non-openssh version of ssh.
>
> Know your options.
>
>
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

[BLUG] scp -p preserving ownership through root user

I have some files that are owned on machine A by user X. As the root
user on machine A, I run

scp -p files root@machineB

The mode and modification times are preserved, but the ownership is
not. I know that with rsync -a the ownership is preserved, but
according ot the scp man page, ownership isn't preserved with -p. I
just got burned by this. Tell me if I'm insane or not but I could swear
that this worked before. It might have had this in a much older version
of scp, like the old non-openssh version of ssh.

Know your options.

--
Mark Krenz
Bloomington Linux Users Group
http://www.bloomingtonlinux.org/
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug