We’ve all had days like this. Someone makes a bone headed mistake and wipes out a ton of work. Our faithful backup systems were working as planned but the backups were corrupt. I know I’ve been through this before. I’m sure you have too. Here’s an interesting video on Pixar’s experience and just how lucky they were to have survived it relatively unscathed.
Posts Tagged ‘Security’
I found an interesting article today studying the frequency of PIN usage on the iPhone. If you have an easily guessable PIN number then you really are doing yourself a disservice. I’d bet that people who use these common PIN numbers on their iPhones use them everywhere else as well. If you find yourself in this boat then you should consider changing it up!
Naturally, 1234 is the most common passcode: mimicking the most common internet passwords. To put this into perspective, these 10 codes represent 15% of all passcodes in use. Most of the top passcodes follow typical formulas, such as four identical digits, moving in a line up/down the pad, repetition. 5683 is the passcode with the least obvious pattern, but it turns out that it is the number representation of LOVE (5683), once again mimicking a very common internet password: “iloveyou.”
Interestingly, 1990-2000 are all in the top 50, and 1980-1989 are all in the top 100. I would interpret this occurrence as a subset of users that set their passcodes to the year of their birth or graduation.
Occasionally at work I am called upon to pick passwords for my end users. They would prefer that I use passwords like “password” or “1234567890.” We all know how secure that is. I prefer to randomly generate a password for them. I have a website bookmarked that I use to generate a few passwords at a time to get me past any immediate needs.
Today I needed to generate 1350 passwords. My favorite tool generates them 50 at a time. What is an IT guy to do? The first thing is to start Googling around. After a brief search I came across a linux utility called makepasswd. After a few command line entries and some brief cleanup in a text editor I was ready to go!
To Install makepasswd (in Ubuntu):
sudo apt-get install makepasswd
To generate the passwords:
makepasswd --chars 8 --count 1350 | cat >> /path/to/file/passwords.txt
The end result is a quick and dirty file of passwords ready to be handed out. These may not be the most secure but they are certainly several orders of magnitude more secure than if I let my folks pick their own. I think I’ll dump the website generator from now on. I have an easy to use and powerful Linux utility right at my fingertips!
Our clergy e-mail server at work is running a Thawte SSL123 certificate for securing all webmail, POP/SSL and SMTP/SSL traffic. They have been an excellent certificate authority and we have used them for several years now. Unfortunately, the certificate in question expired over the weekend. I was supposed to renew it late Saturday night or Sunday morning, the time when we have the least traffic on our server (go figure, it’s a bunch of preachers). The problem was that I got home, sat on the couch, played with the kids and this slipped my mind.
There’s nothing better than a dose of high octane stress to kick off a week just right. I came in this morning to find out that no one could connect to the server anymore. Our office was flooded with calls wondering what was wrong with the e-mail server. In the past people could have clicked past the expiration error and kept on trucking. I quickly renewed the certificate, downloaded and installed it. Then the real problem started. All of our clients could now access the server but they were getting trust errors. Turns out the new Thawte certificates need to have an intermediate CA certificate installed.
Thawte uses Intermediate CAs to enhance the security of SSL and Code Signing certificates. Installing the correct Intermediate CAs or CA bundle for the certificate being used is absolutely essential to ensure that users don’t see certificate errors when visiting a website or running software secured with a Thawte certificate.
I didn’t know about this since it had changed in the last year. After running it by their technical support (they give great chat by the way) I was pointed to an article discussing the issue. Turns out this new requirement was implemented on June 27th, 2010. I downloaded the required certificate and added the following line to ssl.conf:
One quick Apache restart and all is well. Now it’s noon on Monday. Time to get the week started!
Leave it to Google to come up with interesting new things. I downloaded the latest version of Chrome and noticed that several things have changed. You can now synchronize extensions and preferences across browsers. The best part though is the new graphic for SSL certificates:
This is the graphic that you now see when you are visiting a website with a valid certificate that is loading page elements that are not encrypted. It’s not a terrible thing security wise but nevertheless, beware the red skull of security! Even though I know it is going to generate support calls I still love it. Well done Google!
I have noticed over the last few days that we have had Roadrunner messages in our main e-mail gateway stuck in our queue with the following error:
56FE71501A1C 1695 Fri Jul 30 08:32:19 firstname.lastname@example.org
(host hrndva-smtpin02.mail.rr.com[188.8.131.52] refused to talk to me: 421 4.7.1 - Connection refused - - Too many concurrent connections from source IP)
Due to Roadrunner’s rather draconian inbound rate limit policy, we receive this bounce error almost constantly. A fairly large number of the e-mail that is sent out over our servers is headed for Roadrunner’s system. Given that much of the traffic is generated from local church mailing lists, a delay of a few hours usually isn’t that big of a deal. At least, it hasn’t been up until this week. Once I got in this morning however, I saw that we had almost 5,000 messages in the queue with the same error. Alarm bells went off and the IT Office shifted into DEFCON 1.
Searching through the 5,000 messages quickly showed me that there was one sender who was doing most of the work. Grepping through /var/log/maillog quickly showed me which server the traffic was coming from. Once I had this information handy I shut down the main gateway server to stop the bleeding until I could figure out what was going on. I headed over to the server in question and started searching through those logs as well.
Turns out that there was an old Squirrelmail plugin that the spammer was using to generate plain text spam messages internally. Specifically these were nigerian phishing scheme messages. Since they are plain text, coming to and from legitimate addresses and were constantly changing, they were very difficult for our spam filters to stop. Once I updated the plugins, cleaned out all of the mail queues in question and restarted the affected services we were back in action.
The only problem remaining was to convince my fellow e-mail administrators that we were no longer sending spam. All of the major ISP’s use different filtering systems, real time blacklists (RBL’s) and their own internal policies. This meant visiting each ISP listed in the queue as a bounced message and reporting to them individually that the issue was resolved.
Along the way I noticed that there were two agencies that many of the ISP’s use as a clearing house for online e-mail system reputation. You can sign up your organization and they will independently verify that you are not a spammer. Once you are on their list, the ISP’s that subscribe to their services can verify that you are a legitmate sender. It amounts to a shake down more or less. You have to pay to play. Given the importance of our e-mail systems I decided to go ahead and sign us up. If you need to do the same with your mail servers then visit these companies:
- E-mailReg.org – This service charges $20 for one year of service. After verifying domain ownership they say it will take several days.
- ReturnPath.net – This is the big service that a lot of the major ISP’s use. They charge for their service based on how big your organization is. We received the non-profit discount and paid $200 for the application fee with no monthly fee. This system will take several weeks to verify. You really should sign up for this service when times are good.
All is well now. The spam flow has been stopped and all of our queues are cleaned out. All we can do now is wait on Roadrunner’s rate limits to time out and allow us to resume sending messages.
I have been spending a good deal of time in our mailing list server archives trying to run down several permissions related problems. After doing a great deal of searching online I realized that there was no place online that listed the comprehensive required permissions for the /var/lib/mailman/archives and /var/lib/mailman/lists folders. I spent a few hours today blindly stumbling through the permissions before I got them right so I thought I would print them here for reference. This is by no means a comprehensive list of the official permissions. It is however, what is working for me.
drwxrwsr-x 50 root mailman 4.0K Jul 26 13:17 . drwxrwx--- 312 root mailman 20K Jul 29 14:04 .. drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-April -rw-rw-r-- 1 root mailman 13K Jul 29 13:35 2010-April.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-February -rw-rw-r-- 1 root mailman 8.7K Jul 29 13:35 2010-February.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-January -rw-rw-r-- 1 root mailman 21K Jul 29 13:35 2010-January.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-July -rw-rw-r-- 1 root mailman 34K Jul 29 13:35 2010-July.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-June -rw-rw-r-- 1 root mailman 25K Jul 29 13:35 2010-June.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-March -rw-rw-r-- 1 root mailman 24K Jul 29 13:35 2010-March.txt drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 2010-May -rw-rw-r-- 1 root mailman 22K Jul 29 13:35 2010-May.txt drwxrwxr-x 569 root mailman 20K Jul 29 13:35 attachments drwxrwx--- 2 root mailman 24K Jul 29 13:36 database -rw-rw-r-- 1 root mailman 38K Jul 29 13:36 index.html -rw-rw---- 1 root mailman 2.7K Jul 29 13:36 pipermail.pck
drwxrwxr-x 2 root mailman 4.0K Jul 29 13:36 . drwxrwxr-x 94 root mailman 12K Jul 29 13:36 .. -rw-rw-r-- 1 root mailman 2.5K Jul 29 13:36 002505.html -rw-rw-r-- 1 root mailman 2.2K Jul 29 13:36 002506.html -rw-rw-r-- 1 root mailman 2.5K Jul 29 13:36 002507.html -rw-rw-r-- 1 root mailman 4.4K Jul 29 13:36 author.html -rw-rw-r-- 1 root mailman 4.4K Jul 29 13:36 date.html lrwxrwxrwx 1 root mailman 11 Jul 29 13:35 index.html -> thread.html -rw-rw-r-- 1 root mailman 4.4K Jul 29 13:36 subject.html -rw-rw-r-- 1 root mailman 5.1K Jul 29 13:36 thread.html
drwxrwx--- 2 root mailman 24K Jul 29 13:36 . drwxrwxr-x 94 root mailman 12K Jul 29 13:36 .. -rw-rw---- 1 root mailman 31K Jul 29 13:36 2010-July-article -rw-rw---- 1 root mailman 4.4K Jul 29 13:36 2010-July-author -rw-rw---- 1 root mailman 3.9K Jul 29 13:36 2010-July-date -rw-rw---- 1 root mailman 4.6K Jul 29 13:36 2010-July-subject -rw-rw---- 1 root mailman 3.9K Jul 29 13:36 2010-July-thread
drwxrwsr-x 2 root mailman 4.0K Jul 29 13:17 . drwxrwsr-x 194 root mailman 12K Jul 6 21:51 .. -rw-rw---- 1 root mailman 1.7K Jul 6 21:51 admindbpreamble.html -rw-rw---- 1 root mailman 8.9K Jul 6 21:51 config.db -rw-rw---- 1 root mailman 8.9K Jul 6 21:51 config.db.last -rw-rw---- 1 apache mailman 14K Jul 29 13:17 config.pck -rw-rw---- 1 mailman mailman 14K Jul 29 00:54 config.pck.last -rw-rw---- 1 root mailman 12K Jul 27 18:42 digest.mbox -rw-rw---- 1 root mailman 189 Jul 6 21:51 handle_opts.html -rw-rw---- 1 root mailman 1.1K Jul 6 21:51 headfoot.html -rw-rw---- 1 root mailman 3.1K Jul 6 21:51 listinfo.html -rw-rw---- 1 root mailman 4.1K Jul 6 21:51 options.html -rw-rw---- 1 mailman mailman 46 Jul 29 00:54 pending.pck -rw-rw---- 1 root mailman 2 Jul 6 21:51 request.db -rw-rw---- 1 mailman mailman 13K Jul 6 21:51 request.pck -rw-rw---- 1 root mailman 1.2K Jul 6 21:51 roster.html -rw-rw---- 1 root mailman 198 Jul 6 21:51 subscribe.html
After setting these permissions the mailman server resumed normal operations. It looks like apache will take over the files that are edited directly from the web interface. That should be ok. The main problem is giving mailman read/write access to the files that it needs to update and maintain the mailing list archives. Trust me, if mailman can’t access any of these files it will move the message quietly over to the /var/spool/mailman/shunt directory. Nobody wants that. Once you resolve any permissions issues be sure to restart the mailman daemon. To remove e-mail from the shunt directory run /usr/lib/mailman/bin/unshunt.
As I was navigating the internet today at work I noticed a new Google feature that seems rather interesting:
Google has released the beta version of their https encrypted search website. Google probably didn’t encrypt their searches from the beginning due to the increased overhead of the https protocol. After several test searches I cannot tell a difference between the http equivalent. Now we can all search whatever we want in the coffee shop without much worry of wireless sniffers watching our every move. Very impressive!
Google is taking a courageous stand against China. Good for them!
We launched Google.cn in January 2006 in the belief that the benefits of increased access to information for people in China and a more open Internet outweighed our discomfort in agreeing to censor some results. At the time we made clear that “we will carefully monitor conditions in China, including new laws and other restrictions on our services. If we determine that we are unable to achieve the objectives outlined we will not hesitate to reconsider our approach to China.”
These attacks and the surveillance they have uncovered–combined with the attempts over the past year to further limit free speech on the web–have led us to conclude that we should review the feasibility of our business operations in China. We have decided we are no longer willing to continue censoring our results on Google.cn, and so over the next few weeks we will be discussing with the Chinese government the basis on which we could operate an unfiltered search engine within the law, if at all. We recognize that this may well mean having to shut down Google.cn, and potentially our offices in China.