Friday, July 13, 2007

Re: [BLUG] Unix conventions for controlling file access

On Fri, Jul 13, 2007 at 04:16:50PM GMT, Dave Monnier REN-ISAC [dmonnier@ren-isac.net] said the following:
>
> So I wonder, how does this come into play with reserve blocks, quotas, etc?
> I wonder how much stuff depends on intact metadata.

Fortunately, it doesn't seem to be affected by it. At least in Linux
2.2.19 (machine I have access to that has quotas turned on):

Here is a program I quickly wrote in perl to do this:

-------------------------------------------------------------------------------
#!/usr/bin/perl -w

# This is a test to see if I can defeat quota limits by deleting a file
# with an open file handle.

open(FILE, ">/home/markk/largefile");

$i = 0;
$limit = 4600000;
while ($i < $limit) {
print FILE "0";
$i++;
}

sleep 10;
unlink('/home/markk/largefile');

sleep 300;
-------------------------------------------------------------------------------

$ ./test.pl &
[1] 2012
$ ls -l largefile
-rw-r--r-- 1 markk business 393216 Jul 13 13:42 largefile
$ ls -lh largefile
-rw-r--r-- 1 markk business 4.2M Jul 13 13:43 largefile
$ ls -lh largefile
-rw-r--r-- 1 markk business 4.4M Jul 13 13:43 largefile
$ quota -v
Disk quotas for user markk (uid 4505):
Filesystem blocks quota limit grace files quota limit
/dev/sdb1 4640 5000 5500 27 0 0
$ ls -lh largefile
ls: largefile: No such file or directory
$ quota -v
Disk quotas for user markk (uid 4505):
Filesystem blocks quota limit grace files quota limit
/dev/sdb1 4640 5000 5500 27 0 0
$ dd if=/dev/zero of=anotherfile bs=1M count=1000
/home: write failed, user disk quota exceeded too long.
/home: write
failed, user disk quota exceeded too long.
dd:
anotherfile: Disk quota exceeded
1+0 records in
0+0 records out

$ jobs
[1]+ Running ./test.pl &
$ kill %
[1]+ Terminated ./test.pl
$
$ quota -v
Disk quotas for user markk (uid 4505):
Filesystem blocks quota limit grace files quota limit
/dev/sdb1 472 5000 5500 27 0 0

I'm guessing that this also doesn't work in 2.6, but probably checking
is not a bad idea. It would be interesting if it did work and you could
somehow write a program that used FUSE to have a hidden filesystem
inside a deleted file that exceeded your quota.


--
Mark Krenz
Bloomington Linux Users Group
http://www.bloomingtonlinux.org/
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

On Fri, 2007-07-13 at 12:16 -0400, Dave Monnier REN-ISAC wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Mark Krenz wrote:
>
> > Its interesting to note that mysql seems to do this too:
>
> Wow, sure enough.
>
> So I wonder, how does this come into play with reserve blocks, quotas, etc?
> I wonder how much stuff depends on intact metadata.
>

Quotas, block counts, etc are all handled at the inode level -- not the
directory level. So they're just normal files. They just don't have a
name in the filesystem.

You can play around with it by doing something like this in perl:
-------------------------------------------
#!/usr/bin/perl

open(H,">foo");
print H "hello world!\n";
dumpstat("open filehandle",H);
link("foo","foo.link");
dumpstat("two links to filehandle",H);
unlink("foo");
dumpstat("foo is now gone",H);
unlink("foo.link");
dumpstat("foo.link is now also gone",H);
# the file still exists and has data in it
close(H);
# the OS now cleans up, since there are no links and
# the only process with it open has closed it.

sub dumpstat {
my($message,$thing)=@_;
my($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,
$atime,$mtime,$ctime,$blksize,$blocks)
= stat(*$thing);
print $message,"\n";
print " inode: $ino, nlink: $nlink\n";
}
------------------------------

The filesystem uses the inode as the basis for all file operations. The
names are just syntactic sugar for us mere mortals! There's a couple of
side effects of this design decision:

* Each file (the contents & metadata) is uniquely identifiable by a
pair: (device number, inode number). On the above perl script (and
using the 'stat' program) we can see this information:

File: `inode_fun'
Size: 549 Blocks: 16 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 4718621 Links: 1
Access: (0664/-rw-rw-r--) Uid: (11907/bdwheele) Gid: ( 500/bdwheele)
Access: 2007-07-13 12:34:07.000000000 -0400
Modify: 2007-07-13 12:34:03.000000000 -0400
Change: 2007-07-13 12:34:03.000000000 -0400

The only thing that's in the directory entry is the name 'inode_fun' and
the inode number. The 'Device' part comes from the OS, and everything
else comes from the inode.

* any number of names can point to a single set of data. Unlike DOS
complaining about "cross-linked files" when running chkdsk, having
multiple names for a file is a feature.

* mv only changes the name by adding a new link in the destination and
removing the old link in the source. No data is copied. This is why on
early versions of unix you couldn't mv across filesystem boundaries (the
inode numbers are not unique across devices). GNU's mv handles this
case by making a copy.

* The modify and change times, which seem redundant, actually measure
different things. The change time (ctime) is when the inode (meta) data
was changed: ownership, permissions, size, etc. Modification time
(mtime) is when the content was changed. Usually changing the content
changes the file time, so mtime and ctime are the same, but that's not
necessarily true.

* that goofy remove a file while its open thing works :)

* if you don't create enough inodes at filesystem creation time, you
could end up where you couldn't create a new file even though there were
disk blocks free. Its not a big deal these days, but it happened to me
on floppy disks a couple of times, since you want to conserve as much
space as possible by not "wasting" space for the inode tables. df -i
will give you the inode statistics.


That's all I can think of off the top of my head. Hopefully it put
everyone into a dreamy Friday afternoon sleep!

Brian

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

On Fri, 2007-07-13 at 11:56 -0400, Dave Monnier REN-ISAC wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Brian Wheeler wrote:
>
> >> Corrupted how?
>
> I'm sure the sky is the limit. It's a file like any other that's had its
> metadata removed from the directory no? Sounds like a bad idea in general.
> To each their own obviously.
>

I'm not really defending the practice :) I don't like it when processes
do it because disk usage isn't accountable via du and/or ls. But
processes have been doing it for ages.

In any case, the files are ok (and only as corruptible as any other file
is) because the only metadata that's removed is the name and the inode
number (that's all that's in a directory entry). The "important stuff"
like ownership, permissions, size, block lists, number of links, etc are
all stored in the inode itself.

When the number of links is 0 and there are no more processes using the
inode, then the blocks are deallocated. But up until that point, the
inode (and its data) are just like every other file, so its not any more
dangerous than any other file.


Brian


_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Krenz wrote:

> Its interesting to note that mysql seems to do this too:

Wow, sure enough.

So I wonder, how does this come into play with reserve blocks, quotas, etc?
I wonder how much stuff depends on intact metadata.

Thanks Mark, I'd never noticed this done on purpose before.

- -Dave
- --

| Dave Monnier - dmonnier@ren-isac.net |
|

http://nicholas.ren-isac.net/dmonnier/

|
| Principal Security Engineer, REN-ISAC http://www.ren-isac.net/ |
| 24x7 Watch Desk: +1(317)278-6630, ren-isac@ren-isac.net |

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGl6VyBIf6jlONJjIRAvlmAJ4kFCDduRmmz85uBNW9C49GrwADvACeIjVa
4bTaHH449x90Ld1b4oHR6d0=
=riba
-----END PGP SIGNATURE-----
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

I agree with Dave, this does sound like security theatre. However,
according to someone in #kernel on freenode, this technique is decades
old, so it predates a lot of security practices. It doesn't mean its a
good idea though. It could just be that sloppy programmers are not
closing their files before unlinking them.

I've never seen it before and it seemed to be cause of a problem so I
was ready to blame it. I'm still not sure.

I think from a system adminsitrators point of view, it seems like a
really stupid idea, but from a programmers point of view, it seems like
a great idea. Just another example of how those two mindsets really are
different.

Its interesting to note that mysql seems to do this too:

# lsof | grep deleted
[snip]
mysqld 31958 mysql 7u REG 9,0 0 15 /tmp/ibwZfdKe (deleted)
mysqld 31958 mysql 13u REG 9,0 0 16 /tmp/ibR0K4tQ (deleted)
mysqld 31959 mysql 6u REG 9,0 0 14 /tmp/ibuPNjWc (deleted)
mysqld 31959 mysql 7u REG 9,0 0 15 /tmp/ibwZfdKe (deleted)
mysqld 31959 mysql 13u REG 9,0 0 16 /tmp/ibR0K4tQ (deleted)

And you can also still access such deleted files by going into /proc/<pid>/fd

Mark

On Fri, Jul 13, 2007 at 03:33:31PM GMT, Brian Wheeler [bdwheele@indiana.edu] said the following:
> >
> > Sounds like tomfoolery to me. I'd say this qualifies as another act of
> > security theater. I wonder why they would go for this card trick over using
> > actual access control methods.
> >
>
> Well, access controls aside, it is useful for temporary data that you
> definitely want to go away after the process stops. And since multiple
> processes can inherit file descriptors, children of the process that
> opened it can all access it, but others cannot.
>
> > This also sounds like a great way to have their file corrupted.
> >
>
> Corrupted how? Its a file like any other, except the inode has a
> reference count of 0 (+ the open process(es) using it) so its not in any
> danger of being overwritten by other filesystem activity.
>
>
> Brian
>
>
> > - -Dave

--
Mark Krenz
Bloomington Linux Users Group
http://www.bloomingtonlinux.org/
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian Wheeler wrote:

>> Corrupted how?

I'm sure the sky is the limit. It's a file like any other that's had its
metadata removed from the directory no? Sounds like a bad idea in general.
To each their own obviously.

- -Dave

- --

| Dave Monnier - dmonnier@ren-isac.net |
|

http://nicholas.ren-isac.net/dmonnier/

|
| Principal Security Engineer, REN-ISAC http://www.ren-isac.net/ |
| 24x7 Watch Desk: +1(317)278-6630, ren-isac@ren-isac.net |

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGl6CkBIf6jlONJjIRAqZZAKDHnc72LZ3sj3QInYGm7X2vwsOH+wCfdGQ7
pZ/dOV/pk6mmuSRz10PaI80=
=rah5
-----END PGP SIGNATURE-----
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

On Fri, 2007-07-13 at 11:29 -0400, Dave Monnier REN-ISAC wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Mark Krenz wrote:
> > I'm curious to get opinions on this. I'm working with some support
> > technicians for a software company that shall remain nameless for now.
> >
> > Anyways, a problem came up where temporary files are being written to
> > /tmp by a program and then showing up in the output of lsof as
> > (deleted). When talking with the support people about this, they said
> > that their method of controlling access to these files is to create
> > them, then unlink the file while the program is still running. This
> > makes the entry disappear from the directory listing. The program
> > should still be able to access the file when the filehandle is open, but
> > for the most part other programs cannot access it.
> >
> > What do you think about this?
> >
>
> Sounds like tomfoolery to me. I'd say this qualifies as another act of
> security theater. I wonder why they would go for this card trick over using
> actual access control methods.
>

Well, access controls aside, it is useful for temporary data that you
definitely want to go away after the process stops. And since multiple
processes can inherit file descriptors, children of the process that
opened it can all access it, but others cannot.

> This also sounds like a great way to have their file corrupted.
>

Corrupted how? Its a file like any other, except the inode has a
reference count of 0 (+ the open process(es) using it) so its not in any
danger of being overwritten by other filesystem activity.


Brian


> - -Dave
> - --
>
> | Dave Monnier - dmonnier@ren-isac.net |
> |

http://nicholas.ren-isac.net/dmonnier/

|
> | Principal Security Engineer, REN-ISAC http://www.ren-isac.net/ |
> | 24x7 Watch Desk: +1(317)278-6630, ren-isac@ren-isac.net |
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFGl5pMBIf6jlONJjIRAlwsAJ4zBR9PvUCt1plMLZ8Bsn/f2C+O6gCeLXIA
> U+0VBgeKa20Iyztn6i3E/hI=
> =UxI6
> -----END PGP SIGNATURE-----
> _______________________________________________
> BLUG mailing list
> BLUG@linuxfan.com
> http://mailman.cs.indiana.edu/mailman/listinfo/blug

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Krenz wrote:
> I'm curious to get opinions on this. I'm working with some support
> technicians for a software company that shall remain nameless for now.
>
> Anyways, a problem came up where temporary files are being written to
> /tmp by a program and then showing up in the output of lsof as
> (deleted). When talking with the support people about this, they said
> that their method of controlling access to these files is to create
> them, then unlink the file while the program is still running. This
> makes the entry disappear from the directory listing. The program
> should still be able to access the file when the filehandle is open, but
> for the most part other programs cannot access it.
>
> What do you think about this?
>

Sounds like tomfoolery to me. I'd say this qualifies as another act of
security theater. I wonder why they would go for this card trick over using
actual access control methods.

This also sounds like a great way to have their file corrupted.

- -Dave
- --

| Dave Monnier - dmonnier@ren-isac.net |
|

http://nicholas.ren-isac.net/dmonnier/

|
| Principal Security Engineer, REN-ISAC http://www.ren-isac.net/ |
| 24x7 Watch Desk: +1(317)278-6630, ren-isac@ren-isac.net |

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGl5pMBIf6jlONJjIRAlwsAJ4zBR9PvUCt1plMLZ8Bsn/f2C+O6gCeLXIA
U+0VBgeKa20Iyztn6i3E/hI=
=UxI6
-----END PGP SIGNATURE-----
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

Re: [BLUG] Unix conventions for controlling file access

On Fri, 2007-07-13 at 15:13 +0000, Mark Krenz wrote:
> I'm curious to get opinions on this. I'm working with some support
> technicians for a software company that shall remain nameless for now.
>
> Anyways, a problem came up where temporary files are being written to
> /tmp by a program and then showing up in the output of lsof as
> (deleted). When talking with the support people about this, they said
> that their method of controlling access to these files is to create
> them, then unlink the file while the program is still running. This
> makes the entry disappear from the directory listing. The program
> should still be able to access the file when the filehandle is open, but
> for the most part other programs cannot access it.
>
> What do you think about this?
>

Its not unheard of, though it is a bit annoying if you're trying to
figure out who is sucking up all the disk. The advantage to do it this
way is there is no other way to easily open the file (There's an ioctl()
call using I_OPEN which might do it) and it is automatically dallocated
when the application shuts down (regardless if its a nice shutdown or
not).

Brian

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

[BLUG] Unix conventions for controlling file access

I'm curious to get opinions on this. I'm working with some support
technicians for a software company that shall remain nameless for now.

Anyways, a problem came up where temporary files are being written to
/tmp by a program and then showing up in the output of lsof as
(deleted). When talking with the support people about this, they said
that their method of controlling access to these files is to create
them, then unlink the file while the program is still running. This
makes the entry disappear from the directory listing. The program
should still be able to access the file when the filehandle is open, but
for the most part other programs cannot access it.

What do you think about this?

--
Mark Krenz
Bloomington Linux Users Group
http://www.bloomingtonlinux.org/
_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug