Planet Debian Administration

3rd August 2016

dkg: Changes for GnuPG in Debian

Tags:

The GNU Privacy Guard (GnuPG) upstream team maintains three branches of development: 1.4 ("classic"), 2.0 ("stable"), and 2.1 ("modern").

They differ in various ways: software architecture, supported algorithms, network transport mechanisms, protocol versions, development activity, co-installability, etc.

Debian currently ships two versions of GnuPG in every maintained suite -- in particular, /usr/bin/gpg has historically always been provided by the "classic" branch.

That's going to change!

Debian unstable will soon be moving to the "modern" branch for providing /usr/bin/gpg. This will give several advantages for Debian and its users in the future, but it will require a transition. Hopefully we can make it a smooth one. What are the benefits?

Compared to "classic", The "modern" branch has:

If you want to try this out, the changes are already made in experimental. Please experiment! What does this mean for end users?

If you're an end user and you don't use GnuPG directly, you shouldn't notice much of a change once the packages start to move through the rest of the archive.

Even if you do use GnuPG regularly, you shouldn't notice too much of a difference. One of the main differences is that all access to your secret key will be handled through gpg-agent, which should be automatically launched as needed. This means that operations like signing and decryption will cause gpg-agent to prompt the the user to unlock any locked keys directly, rather than gpg itself prompting the user.

If you have an existing keyring, you may also notice a difference based on a change of how your public keys are managed, though again this transition should ideally be smooth enough that you won't notice unless you care to investigate more deeply.

If you use GnuPG regularly, you might want to read the NEWS file that ships with GnuPG and related packages for updates that should help you through the transition.

If you use GnuPG in a language other than English, please install the gnupg-l10n package, which contains the localization/translation files. For versions where those files are split out of the main package, gnupg explicitly Recommends: gnupg-l10n already, so it should be brought in for new installations by default.

If you have an archive of old data that depends on known-broken algorithms, PGP3 keys, or other deprecated material, you'll need to have "classic" GnuPG around to access it. That will be provided in the gnupg1 package What does this mean for package maintainers?

If you maintain a package that depends on gnupg: be aware that the gnupg package in debian is going through this transition.

A few general thoughts:

If you maintain a package that depends on gnupg2 and tries to use gpg2 instead of gpg, that should stay ok. However, at some point it'd be nice to get rid of /usr/bin/gpg2 and just have one expected binary (gpg). So you can help with that:

What specifically needs to happen?

The last major step for this transition was renaming the source package for "classic" GnuPG to be gnupg1. This transition is currently in the ftp-master's NEW queue. Once it makes it through that queue, and both gnupg1 and gnupg2 have been in experimental for a few days without reports of dangerous breakage, we'll upload both gnupg1 and gnupg2 to unstable.

We'll also need to do some triage on the BTS, reassigning some reports which are really only relevant for the "classic" branch.

Please report bugs via the BTS as usual! You're also welcome to ask questions and make suggestions on #debian-gnupg on irc.oftc.net, or to mail the Debian GnuPG packaging team at pkg-gnupg-maint@lists.alioth.debian.org.

Happy hacking!

3rd August 2016 21:55:49 : 10 comments.

9th July 2016

echtap: Is it ever too late to learn?

I sometimes feel learning to be a server adminstrator is sometimes quite intimidating. What do you think should be the most important first-steps you should take in learning how to be a server adminstrator, especially as a newbie looking to improve their skillset.
9th July 2016 15:26:26 : 1 comment.

17th May 2016

rkreider: I still creep

In my last entry, dated Nov 23 2014, I informed the world I'd be more active. Well, fast forward to 2016 and this constitutes my latest post. Last time I was around, I believe I downloaded and tinkered with Steve's platform. I think I got it working, but that was 2 years ago almost. A familiar little place this is. It is nice to pop back in every once and awhile to see what everyone is posting. It's like Facebook, with less Facebook. -Rich
17th May 2016 03:26:43 : No comments. Link

5th May 2016

Steve: Site overhaul complete ...

Tags: ,

No hugely significant changes, except for the bootstrap-based theme which is still a work-in-progress, and the switch to SSL-everywhere.

5th May 2016 05:00:22 : 4 comments.

27th February 2016

simonw: Logitech Wireless Headset with Jessie

One of those irritations.

Logitech H800 wireless headset, works with Squeeze, works with Mac OS X, works with Windows 10.

Works initially on Jessie with KDE desktop, but I get a weird errors in the Xorg.log and then the sound quality (which starts perfect) starts to degrade (seemingly with events in the UI), and quite quickly degrades to unlistenable. The exact behaviour clearly depends on activity of some sort, but 4 or 5 minutes is not uncommon.

I've tried the xf86-input-evdev from unstable, I've tried building it from source (2.10.1), the weird errors remain.

The errors look like this bug, and I commented to that effect.

https://bugs.freedesktop.org/show_bug.cgi?id=89473#c9

I tried some vague attempts to make Jessie sound work more like Squeeze, on the assumption it was changes to have priority but that makes no difference.

The author however thinks it is unlikely to be responsible for the sound quality issue.

Anyone got any idea where to go next?

[ 57658.500] (EE) BUG: triggered 'if (axnum >= dev->valuator->numAxes)'
[ 57658.500] (EE) BUG: ../../Xi/exevents.c:2086 in InitValuatorAxisStruct()
[ 57658.500] (EE) 
[ 57658.500] (EE) Backtrace:
[ 57658.500] (EE) 0: /usr/bin/X (xorg_backtrace+0x56) [0x7fe4c7d73d46]
[ 57658.500] (EE) 1: /usr/bin/X (InitValuatorAxisStruct+0x67) [0x7fe4c7d02e77]
[ 57658.500] (EE) 2: /usr/lib/xorg/modules/input/evdev_drv.so (0x7fe4ba864000+0x51ca) [0x7fe4ba8691ca]
[ 57658.500] (EE) 3: /usr/lib/xorg/modules/input/evdev_drv.so (0x7fe4ba864000+0x565f) [0x7fe4ba86965f]
[ 57658.500] (EE) 4: /usr/lib/xorg/modules/input/evdev_drv.so (0x7fe4ba864000+0x7393) [0x7fe4ba86b393]
[ 57658.500] (EE) 5: /usr/bin/X (ActivateDevice+0x4a) [0x7fe4c7c093da]
[ 57658.500] (EE) 6: /usr/bin/X (0x7fe4c7bbd000+0xa4b31) [0x7fe4c7c61b31]
[ 57658.500] (EE) 7: /usr/bin/X (0x7fe4c7bbd000+0xbb43b) [0x7fe4c7c7843b]
[ 57658.500] (EE) 8: /usr/bin/X (0x7fe4c7bbd000+0xbba13) [0x7fe4c7c78a13]
[ 57658.500] (EE) 9: /usr/bin/X (config_init+0x9) [0x7fe4c7c77669]
[ 57658.500] (EE) 10: /usr/bin/X (InitInput+0xbb) [0x7fe4c7c553db]
[ 57658.500] (EE) 11: /usr/bin/X (0x7fe4c7bbd000+0x5b559) [0x7fe4c7c18559]
[ 57658.501] (EE) 12: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xf5) [0x7fe4c58d0b45]
[ 57658.501] (EE) 13: /usr/bin/X (0x7fe4c7bbd000+0x4590e) [0x7fe4c7c0290e]
27th February 2016 22:19:29 : No comments. Link

5th February 2016

Steve: Changes ...

Tags: ,

In the near future this site will be undergoing an overhaul:

Partly because I think it'll be a fun experiment, partly because the layout is something I'm just not happy with.

5th February 2016 07:25:01 : 2 comments.

30th January 2016

naresh3410: linuc file system creation

please help me to create a file system in linux/unix in step by step.
30th January 2016 16:06:25 : Comments disabled

20th December 2015

ajt: Debian surprises

At the start of this month I upgraded my last Debian Wheezy box to Jessie. I'd held out for ages, as it was my home server and had more on it than my BigV public server or my desktop systems.

There were two pain points in the end:

That's all my boxes, both real and virtual upgraded. I've also rebuilt my better half's desktop with a new motherboard & graphics card, so it's now got a significantly faster CPU and GPU, plus more RAM and a much faster main bus to play with. I even managed to freecycle the old motherboard, so that's a win all round.

20th December 2015 12:25:23 : 4 comments.

5th December 2015

Steve: SSL Updates

Tags: , ,

We now use Let's Encrypt for:

Future tweaks to come, but bug reports are most welcome.

5th December 2015 05:51:01 : 2 comments.

28th November 2015

Steve: ReCAPTCHA

Tags: , , ,

I've just culled a few thousand spam accounts, and added recaptcha to our signup-page as a quick hack to see if I can prevent this from becoming even more of a problem.

This site has 124,348 registered users. I suspect maybe 2048 are genuine. The rest are bots. le sigh.

28th November 2015 07:20:20 : No comments. Link

9th November 2015

lee: Selective and multiple domain DKIM with Exim

Tags: ,
My previous config from 2011 suggested the use of custom router and transport files for supporting selective domain use for DKIM in exim, but since the Debian package contains "ifdef" for expansions, you can achieve the same effect with lookups. Assuming the correct DNS records have been set up, add the key into /etc/exim4/dkim-foo.key and make it readable by the exim user (Debian-exim). Create /etc/exim4/dkim_senders with a list of addresses that should have mail signed.
test@example.com
*@example.net
Create /etc/exim4/dkim_domains with the per-domain configs
example.com: selector=foo key=/etc/exim4/dkim-foo.key canon=relaxed
example.net: selector=bar key=/etc/exim4/dkim-bar.key
Create /etc/exim4/conf.d/main/00_local_dkim (if you're using split config)
 DKIM_DOMAIN =      ${lookup{$sender_address}lsearch*@{/etc/exim4/dkim_senders}{$sender_address_domain}{}}

 ## make the following active instead if all mail from selected domains should be signed
 # DKIM_DOMAIN =      ${lookup{$sender_address_domain}lsearch*@{/etc/exim4/dkim_domains}{$sender_address_domain}{}}

 DKIM_SELECTOR =    ${extract{selector}{${lookup{$sender_address_domain}lsearch*@{/etc/exim4/dkim_domains}}}{$value}{}}
 DKIM_PRIVATE_KEY = ${extract{key}{${lookup{$sender_address_domain}lsearch*@{/etc/exim4/dkim_domains}}}{$value}{}}
 DKIM_CANON =       ${extract{canon}{${lookup{$sender_address_domain}lsearch*@{/etc/exim4/dkim_domains}}}{$value}{relaxed}}
 DKIM_STRICT =      ${extract{strict}{${lookup{$sender_address_domain}lsearch*@{/etc/exim4/dkim_domains}}}{$value}{false}}
Run update-exim4.conf and reload exim. For addresses not listed in /etc/exim4/dkim_senders exim should not attempt DKIM signing. This config assumes that the signing domain is the sender's domain. It's reasonable, but not necessarily always true. It also assumes users on the same sender domain use the same signing key. If necessary it wouldn't be too hard to swap the lookups around to allow domains to support different selectors.
9th November 2015 11:40:04 : No comments. Link

29th July 2015

Steve: Nowadays I live in Finland

Tags:

My (Finnish) wife and I have now relocated from Edinburgh, Scotland, to Helsinki, Finland.

I've been here for 13 days at the moment, and am still settling in. But so far things are going well. So, yeah, that happened.

29th July 2015 06:39:14 : 2 comments.

4th June 2015

dkg: Challenge: one reproducible package a week

Tags: ,
I encourage anyone interested in debian development to get involved with the Reproducible Builds project. My own project is to try to diagnose (and hopefully provide patches for) two unreproducible packages a week. Maybe you can do one package a week?

Reproducible Builds is another example of the kind of audacious project that I celebrated in my last blog post.

It's a fun way to learn a little bit about different parts of the archive, and to help on an incrementally-improving process. My workflow below is meant to encourage folks to join in. The documentation on the wiki is certainly more authoritative (and will be more up-to-date in the future).

My usual workflow is this:

The information at the R-B wiki page is quite detailed if you want more info.

Finally, many many thanks to all of the people involved in the project, and particularly to h01ger and Lunar^, who have both contributed a ton of work, and have also made it easy to plug in and help as a less-involved contributor. The nice automatically-updated statistics provided by the team's continuous integration work makes it a fun game to help out.

4th June 2015 02:31:12 : No comments. Link

17th May 2015

mcortese: GNOME Wayland

I've just realized that the last version of GNOME (the one shipped with Jessie) allows to start the session under Wayland instead of the classic X. I think it's a only a preview to start collecting some feedback and bug reports from the field, and that it shouldn't be considered a mature, workable option.

On my admittedly underpowered system (an Atom-based netbook) the usability approaches zero, with xwayland taking up 99% of the CPU and several seconds of lag between action (mouse clicks, key presses) and feedback. In addition, synaptics options like tap-to-click are not recognized -- or not obeyed, I'm not sure.

I'd be curious about other users' experience, if any have been brave enough to try it out!

17th May 2015 10:23:00 : 2 comments.

8th May 2015

dkg: Cheers to audacity!

Tags:
When paultag recently announced a project to try to move debian infrastructure to python3, my first thought was how large that undertaking would likely be. It seems like a classic engineering task, full of work and nit-picky details to get right, useful/necessary in the long-term, painful in the short-term, and if you manage to pull it off successfully, the best you can usually hope for is that no one will notice that it was done at all.

I always find that kind of task a little off-putting and difficult to tackle, but I was happy to see someone driving the project, since it does need to get done. Debian is potentially also in a position to help the upstream python community, because we have a pretty good view of what things are being used, at least within our own ecosystem.

I'm happy to say that i also missed one of the other great benefits of paultag's audacious proposal, which is how it has engaged people who already knew about debian but who aren't yet involved. Evidence of this engagement is already visible on the py3porters-devel mailing list. But if that wasn't enough, I ran into a friend recently who told me, "Hey, I found a way to contribute to debian finally!" and pointed me to the py3-porters project. People want to contribute to the project, and are looking for ways in.

So cheers to the people who propose audacious projects and make them inviting to everyone, newcomers included. And cheers to the people who step up to potentially daunting work, stake out a task, roll up their sleeves, and pitch in. Even if the py3porters project doesn't move all of debian's python infrastructure to pyt3 as fast as paultag wants it to, i think it's already a win for the project as a whole. I am looking forward to seeing what comes out of it (and it's reminding me i need to port some of my own python work, too!)

The next time you stumble over something big that needs doing in debian, even something that might seem impossible, please make it inviting, and dive in. The rest of the project will grow and improve from the attempt.

8th May 2015 18:43:51 : No comments. Link

1st May 2015

dkg: Preferred Packaging Practices

Tags: , ,
I just took a few minutes to write up my preferred Debian packaging practices.

The basic gist is that i like to use git-buildpackage (gbp) with the upstream source included in the repo, both as tarballs (with pristine-tar branches) and including upstream's native VCS history (Joey's arguments about syncing with upstream git are worth reading if you're not already convinced this is a good idea).

I also started using gbp-pq recently -- the patch-queue feature is really useful for at least three things:

My preferred packaging practices document is a work in progress. I'd love to improve it. If you have suggestions, please let me know.

Also, if you've written up your own preferred packaging practices, send me a link! I'm hoping to share and learn tips and tricks around this kind of workflow at debconf 15 this year.

1st May 2015 19:41:06 : No comments. Link

16th April 2015

fugit: Building a certain Version of Debian

The Problem: Vendor software we were looking to evaluate did not compile correctly using the lastest version of wheezy(7.8 at the time of writing).

The Solution: We had a to build a new server using snapshot.debian.org. This allowed us to build the server at a point in time after 7.1 was released but prior to the 7.2 release. Below are the settings we use.

    cat /etc/apt/sources.list
    deb     http://snapshot.debian.org/archive/debian/20130802T034916Z/ wheezy main
    deb     http://snapshot.debian.org/archive/debian-security/20130802T202345Z/  wheezy/updates main

For more details head on over to the site.

16th April 2015 16:56:15 : 2 comments.

16th March 2015

dkg: Bootable grub USB stick (EFI and BIOS for Intel)

Tags: , , ,

I'm using grub version 2.02~beta2-2.

I want to make a USB stick that's capable of booting Intel architecture EFI machines, both 64-bit (x86_64) and 32-bit (ia32). I'm starting from a USB stick which is attached to a running debian system as /dev/sdX. I have nothing that i care about on that USB stick, and all data on it will be destroyed by this process.

I'm also going to try to make it bootable for traditional Intel BIOS machines, since that seems handy.

I'm documenting what I did here, in case it's useful to other people.

Set up the USB stick's partition table:

parted /dev/sdX -- mktable gpt
parted /dev/sdX -- mkpart biosgrub fat32 1MiB 4MiB
parted /dev/sdX -- mkpart efi fat32 4MiB -1
parted /dev/sdX -- set 1 bios_grub on
parted /dev/sdX -- set 2 esp on
After this, my 1GiB USB stick looks like:
0 root@foo:~# parted /dev/sdX -- print
Model:  USB FLASH DRIVE (scsi)
Disk /dev/sdX: 1032MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name      Flags
 1      1049kB  4194kB  3146kB  fat32        biosgrub  bios_grub
 2      4194kB  1031MB  1027MB               efi       boot, esp

0 root@foo:~# 
make a filesystem and mount it temporarily at /mnt:
mkfs -t vfat -n GRUB /dev/sdX2
mount /dev/sdX2 /mnt
ensure we have the binaries needed, and add three grub targets for the different platforms:
apt install grub-efi-ia32-bin grub-efi-amd64-bin grub-pc-bin grub2-common

grub-install --removable --no-nvram --no-uefi-secure-boot \
     --efi-directory=/mnt --boot-directory=/mnt \
     --target=i386-efi

grub-install --removable --no-nvram --no-uefi-secure-boot \
     --efi-directory=/mnt --boot-directory=/mnt \
     --target=x86_64-efi

grub-install --removable --boot-directory=/mnt \
     --target=i386-pc /dev/sdX
At this point, you should add anything else you want to /mnt here! For example: And don't forget to cleanup:
umount /mnt
sync
16th March 2015 23:12:35 : 15 comments.

2nd February 2015

ajt: DKIM

Tags: , , , ,

Recently a friend asked me about SPF and DKIM. I was vaguely aware of them and their generally perceived uselessness. I've never bothered with them but thought I'd look into them for him. SPF was trivial to set up using my Bytemark DNS, and easy to testing using of many on-line email tools.

The challenge is DKIM, I found several sites that explained what to do - they were all variation around a theme and seemed easy enough. Generate a key pair (check), put the public half into your DNS (check), tweak Exim4 (ARGH!!!).

As far as I tell I followed what the web site said and reloaded the Exim config and it looks like it's all loaded into place, the key is readable by Exim, the path is correct but no matter what I do Exim wont add the DKIM signature to my outgoing emails...!

2nd February 2015 20:55:28 : No comments. Link

12th December 2014

dkg: a10n for l10n

The abbreviated title above means "Appreciation for Localization" :)

I wanted to say a word of thanks for the awesome work done by debian localization teams. I speak English, and my other language skills are weak. I'm lucky: most software I use is written by default in a language that I can already understand.

The debian localization teams do great work in making sure that packages in debian gets translated into many other languages, so that many more people around the world can take advantage of free software.

I was reminded of this work recently (again) with the great patches submitted to GnuPG and related packages. The changes were made by many different people, and coordinated with the debian GnuPG packaging team by David Prévot.

This work doesn't just help debian and its users. These localizations make their way back upstream to the original projects, which in turn are available to many other people.

If you use debian, and you speak a language other than english, and you want to give back to the community, please consider joining one of the localization teams. They are a great way to help out our project's top priorities: our users and free software.

Thank you to all the localizers!

(this post was inspired by gregoa's debian advent calendar. i won't be posting public words of thanks as frequently or as diligently as he does, any more than i'll be fixing the number of RC bugs that he fixes. This are just two of the ways that gregoa consistently leads the community by example. He's an inspiration, even if living up to his example is a daunting challenge.)

12th December 2014 23:00:04 : 1 comment.

23rd November 2014

rkreider: It's been 7 Years

It's been seven years since I last posted an entry here. WOW! I re-read some of my blog entries and laughed at myself. I've learned so much since then. Maybe I'll try to be a bit more active in the coming future.
23rd November 2014 03:54:14 : 1 comment.

6th November 2014

dkg: GnuPG 2.1.0 in debian experimental

Tags: , ,
Today, i uploaded GnuPG 2.1.0 into debian's experimental suite. It's built for amd64 and i386 and powerpc already. You can monitor its progress on the buildds to see when it's available for your architecture.

Changes

GnuPG 2.1 offers many new and interesting features, but one of the most important changes is the introduction of elliptic curve crypto (ECC). While GnuPG 2.1 discourages the creation of ECC keys by default, it's important that we have the ability to verify ECC signatures and to encrypt to ECC keys if other people are using this tech. It seems likely, for example, that Google's End-To-End Chrome OpenPGP extension will use ECC. GnuPG users who don't have this capability available won't be able to communicate with End-To-End users.

There are many other architectural changes, including a move to more daemonized interactions with the outside world, including using dirmngr to talk to the keyservers, and relying more heavily on gpg-agent for secret key access. The gpg-agent change is a welcome one -- the agent now holds the secret key material entirely and never releases it -- as of 2.1 gpg2 never has any asymmetric secret key material in its process space at all.

One other nice change for those of us with large keyrings is the new keybox format for public key material. This provides much faster indexed access to the public keyring.

I've been using GnuPG 2.1.0 betas regularly for the last month, and i think that for the most part, they're ready for regular use.

Timing for debian

The timing between the debian freeze and the GnuPG upstream is unfortunate, but i don't think i'm prepared to push for this as a jessie transition yet, without more backup. I'm talking to other members of the GnuPG packaging team to see if they think this is worth even bringing to the attention of the release team, but i'm not pursuing it at the moment.

If you really want to see this in debian jessie, please install the experimental package and let me know how it works for you.

Long term migration concerns

GnuPG upstream is now maintaining three branches concurrently: modern (2.1.x), stable (2.0.x), and classic (1.4.x). I think this is stretches the GnuPG upstream development team too thin, and we should do what we can to help them transition to supporting fewer releases concurrently.

In the long-term, I'd ultimately like to see gnupg 2.1.x to replace all use of gpg 1.4.x and gpg 2.0.x in debian, but unlikely to to happen right now.

In particular, the following two bugs make it impossible to use my current, common monkeysphere workflow:

And GnuPG 2.1.0 drops support for the older, known-weak OpenPGPv3 key formats. This is an important step for simplification, but there are a few people who probably still need to use v3 keys for obscure/janky reasons, or have data encrypted to a v3 key that they need to be able to decrypt. Those people will want to have GnuPG 1.4 around.

Call for testing

Anyway, if you use debian testing or unstable, and you are interested in these features, i invite you to install `gnupg2` and its friends from experimental. If you want to be sensibly conservative, i recommend backing up `~/.gnupg` before trying to use it:

cp -aT .gnupg .gnupg.bak
sudo apt install -t experimental gnupg2 gnupg-agent dirmngr gpgsm gpgv2 scdaemon
If you find issues, please file them via the debian BTS as usual. I (or other members of the pkg-gnupg team) will help you triage them to upstream as needed.
6th November 2014 23:27:16 : 10 comments.

31st August 2014

Steve: Site code is public once again.

Tags: , ,

Originally the code behind this site as 100% open, but during the course of some of the earlier migrations it was closed by accident - jumping from CVS to hg, and now to git things moved around a lot.

Nothing was ever intended to be secret, private, or closed, it was just an aspect I've failed to pay attention to.

Anyway the code behind this site is called YAWNS and once again it is open to the public:

31st August 2014 14:55:26 : 2 comments.

8th August 2014

Steve: Code overhaul .. volunteers welcome

Tags:

The code behind this site is badly in need of another overhaul.

At the moment it runs on a set of ropy CGI scripts that are neither efficient nor pleasant.

My current plan is to record the basics from scratch, then port over functionality as time goes on. As part of that I think I'll be coming up with some toy-servers:

With that structure in mind the core of the site can just mediate between requests and the actual backend - without worrying about actual implementation-details.

In terms of features the only thing I think I'm going to remove is the comment-feed RSSs.

There is a feed for the comments on every article - and that feed gets spidered like mad, for articles that are many years old.

I'm open to the idea of collaboration if there are users who wish to help - and the code will be on github in due course.

8th August 2014 10:14:56 : 7 comments.

16th July 2014

Steve: Rejoining the Debian project

I resigned from the debian project a few years ago, but I'm currently going through the process to un-retire.

Fun times.

(I have more free time now that I'm happily married and my wife works as a Doctor in A&E - having a few late-night shifts in local hospitals.)

16th July 2014 10:37:18 : 1 comment.

27th June 2014

Steve: Git-based DNS hosting

Tags: , ,

So I accidentally released a service which will give you resilient, low-latency, DNS-hosting (which uses Amazons route53 on the backend).

If you'd like to use Git to store your DNS-data, and add/update DNS-entries via a simple "git push" then you should check it out:

27th June 2014 16:44:06 : No comments. Link

10th June 2014

lee: Using Amazon SES with Exim (Submission)

Tags: , , ,

A previous entry from 2011 on the subject on using SES and Exim still comes up in web searches. As with the 2011 config, this assumes a standard Debian/Ubuntu exim4 deployment using a split-file config.

SES now allows authenticated relay via port 25, but also submission (port 587). Logically it makes more sense to think of SES as a Submission host since it uses ESMTPSA and makes modifications to the mail in transit, so I explicitly create a submission transport at /etc/exim4/conf.d/transport/40_local_submission

remote_submission:
  debug_print = "T: remote_submission for $local_part@$domain"
  driver = smtp
  port = 587
  hosts_require_auth = *
  hosts_require_tls = *
Then I add in a router which activates based on the sender at /etc/exim4/conf.d/router/180_local_aws-ses
aws_ses:
  debug_print = "R: send_via_ses for mail from $sender_address"
  driver = manualroute
  host_find_failed = freeze
  domains = ! +local_domains
  senders = AWS_SES_SENDER
  transport = remote_submission
  route_list = * AWS_SES_SERVER
The site specific configuration /etc/exim4/conf.d/main/00_local_aws-ses
## sender email addresses to be routed via SES, one-per-line
AWS_SES_SENDER = lsearch*@;/etc/exim4/ses_senders

## nearest SES ingress point
AWS_SES_SERVER = email-smtp.eu-west-1.amazonaws.com
And assuming the standard auth handling is in place, add a line to /etc/exim4/passwd.client
*.amazonaws.com:Yourusername:Yourpassword
Note, these are SMTP specific credentials and not the AWS credentials previously used. Run update-exim4.conf and then restart exim.
10th June 2014 21:35:33 : No comments. Link

3rd June 2014

Steve: Hire Steve ...

If you're looking for a system administrator, who is very very familiar with Debian GNU/Linux, please do consider getting in touch.

I'm an ex-member of the Debian project, and was a member of the Debian Security Team, which involved handling security updates for the distribution. (This came about as a result of my interest in auditing software, a process which lead to the discovery, and fixing, of numerous flaws in popular open-source applications and servers.)

I'm based in Edinburgh, but I have had many years working remotely, and would be happy to repeat that.

As you can see from my prior submissions I'm very familiar with system administration, including (but not limited to):

3rd June 2014 12:46:28 : No comments. Link

15th May 2014

Steve: Promoting my own creations ..

Tags: , ,

In the past I've written and posted articles upon this site about my own software.

Over time I've become a little less keen on doing so, on the basis that people might think it is too Steve-centric. That said I'd always created this site to document things of interest to myself, and the fact that others enjoy them is a good bonus.

In summary I wrote a mail client, it is console-based like mutt, but it has a real scripting language you can use to do fun things (Lua).

You can find details here, should you have any interest:

Relatedly I've been pondering a "Hire Steve" banner on the top of the site. I've resisted the temptation for the moment, but give it a couple more weeks of unemployment and I may well change my mind..

15th May 2014 13:31:32 : 2 comments.

12th May 2014

simonw: Web Application Vulnerability scanners

Been trying out various web application vulnerabilities scanners, both Open Source and Proprietary.

These are tools that will analyse your website, or in some case an instrumented copy of your site, and identify some types of common security flaws, or in other cases simple omissions to use best practice.

My main goal is to find tools which are easy to integrate into a Continuous Integration process, so ideally looking for scanners with minimal user interaction, with a command line driven batch mode, to beef up the current CI process.

I've not reached any conclusions yet, but it is an interesting landscape. Almost everyone agrees more tools are better, because they are all slightly different, there are inevitably bugs that one finds, that another misses. CPU time is a LOT cheaper than developer, or Pen Tester time, so a lot of automation makes sense, however the significant false positive rate means that more tools do make some more work. This scales better than I expected, because the same issues trip up different tools in the same way.

For example the WordPress action "wp-post-comments.php" returns a "500 Server Error" HTTP response code under some circumstances, and lots of tools leap on this proudly noting they have "crashed" your server, when the response header shows otherwise.

But alas WordPress defaults to 500 status codes when people omit required fields on forms, and other common cases. This WordPress annoyance has been lurking for at least 4 years, and the Lead Developers seem in no hurry to stop it emitting error codes suggesting it has crashed just because someone forgot to enter their email address.

https://core.trac.wordpress.org/ticket/10551

Almost all the tools pick up on this, and arguably rightly so, but it isn't a useful find.


Intercepting Proxies

Intercepting Proxies sit between the browser, and website to test, and thus can easily identify (and modify if needed) any traffic from browser to server. For AJAX, web sockets, and a number of other web technologies this is an essential place to be for spotting certain common vulnerabilities (like failure to validate AJAX server side).

One of our tools of choice is BurpSuite, which is a fantastic little application. You do need to buy the Pro version to do proper automated scanning. Whilst we'll use it for manual, and pre-release testing, its focus is on manual testing, and whilst some folk have tried to drive it from the command line, I suspect down this route lies pain and eternal maintenance.

The OWASP Zed Attack proxy tool is targeted at almost exactly the same space as BurpSuite (Clone?). In true Open Source style it is harder to use, a little rougher around the edges, but allegedly does more, and seems to have a little more momentum and adaptability. Oh and a very responsive lead funded by Mozilla. So far it takes longer to run, and has more false positives, but in my initial tests identify a whole host of minor but legitimate issues, although I rushed it, it still took nearly 4 hours for a small website (CPU time may be cheap but we do want answers before the release date). I suspect many folk who've never used BurpSuite Pro will think ZAP fantastic, and it does look like a little attention to settings will pay dividends in both time to run, and false positive rate.

I suspect either tools can be used in Continuous Integration as a proxy, but I see more effort to support this from the OWASP ZAP lead.


Vulnerability Scanners

In a previous roles where I was working on PHP and Perl websites, predominantly doing simple forms, the tool of choice was Wapiti. My reading around suggests it is still an excellent choice, scoring well in comparisons and being quick and easy to use. The default version of Wapiti in Debian is 1 until Jessie, so you probably want either a dedicated network security distro like Kali in a machine somewhere, or just Debian testing.

I note also that OSSIM vulnerability scanner (OpenVAS) will run wapiti if it is installed, but again because it is based on Squeeze it is wapiti version 1 you get if you use "apt-get". Still if you need a network security scanner installing OSSIM is a lot easier than installing OpenVAS in Debian, sticking in wapiti is a no-brainer as a one line enhancement. OSSIM can run Nikto with a really minor tweak I've discussed on their forum as well.

Nikto still finds a place in my arsenal. Somehow I can't love w3af. Few of the other tools make me love them (except NMAP of course).

Next on my list are some of the more established proprietary tools, but there are a huge number of Open Source tools I simply won't have time to evaluate, so any tips on where to check first appreciated. I don't see an alternative to the Intercepting proxy, or something logically equivalent, there is also a requirement to handle newer features of browsers like Web Sockets and local storage (which I see as a greatly enriched version of cookie poisoning).

As always these tools won't keep you safe, but they may let you know when you are heading in the wrong direction, which is why using them as early in the development process as possible makes sense.
12th May 2014 21:32:39 : No comments. Link