?

Log in

Agnus Dei's Journal

> recent entries
> calendar
> friends
> profile
> previous 20 entries

Tuesday, November 15th, 2016
7:36 pm - cool sed trick
Wrote something cool tonight.

I like this sed trick. I need to use it more. It basically uses the date stamp string to match that string and until the end of file. So you are only grepping from the first instance of that, until the end of file.

And the date string is set to now minus 10 minutes and then I remove the last character. So if it's currently "2016-11-16 00:18" then it looks from "2016-11-16 00:0" down until end of file.

So the window is always going to be the last 10-20 minutes of logs that it looks at.

ERROR_STRING="com.amazonaws.AmazonClientException: Unable to execute HTTP request: Timeout waiting for connection from pool"
DATE_REGEX=$(date "+%Y-%m-%d %H:%M" -d "10 min ago" | sed s'/.$//')  # Example 2016-11-16 00:0
COUNT=$(sed -n "/$DATE_REGEX/,\$p" ~tomcat/logs/mytomcat.log | grep "$ERROR_STRING" | wc -l )

if [ "$COUNT" -gt "0" ]; then
	do_alert_action
fi


(comment on this)

Monday, November 14th, 2016
1:34 pm - A break down of S3 failed connections from tomcat broken down by count per hour.
A break down of S3 failed connections from tomcat broken down by count per hour.

awk 'BEGIN{RS="\n2016";ORS="\n\n2016";FS=OFS="\n"}/com.amazonaws.AmazonClientException: Unable to execute HTTP request: Timeout waiting for connection from pool/' ~tomcat/logs/mytomcat.log.2016-11-12 | grep -o '....-..-.. ..:' | sort | uniq -c
    406 2016-11-12 16:
    784 2016-11-12 17:
    379 2016-11-12 18:

(comment on this)

Wednesday, September 28th, 2016
3:35 pm - Convert mysql timestamp to human readable date
tail -10000 mysql-slow.log |awk -F'SET timestamp=' 'NF > 1{ system("date -d @" $2) }{print $_}' > foo.log


That will insert a line before the "SET timestamp=" line of the actual date for the query.

(comment on this)

3:34 pm - List the all the licenses installed on your Linux server

List the all the licenses you are using in your distribution of Linux and group them and count them, with the most used on top. :)

for i in `rpm -qa --queryformat '%{name}\n'`; do echo -n "$i:"; rpm -qi $i | grep ^License | head -1 ; echo ; done | awk 'NF' | cut -d: -f3 | sort | uniq -c | sort -rn

(comment on this)

Sunday, August 14th, 2016
8:29 pm - How to Disable SELinux (and remove the "dot" permissions)

Disabling SELinux

Check/Verify the SELinux current status


Run the following command:

sestatus

Example:


[root@osboxes]# sestatus
SELinux status:                 enabled

Disable SELinux on the server

Run the following commands:

sed -i "/SELINUX/  s/enforcing/disabled/" /etc/selinux/config 
reboot

Confirm SELinux is disabled


[root@osboxes]# sestatus
SELinux status:                 disabled

View SELinux permissions

SELinux context remains associated with files regardless whether or not SELinux is enabled.

If you want to see the actual SELinux context associated with the files you can use either of following commands


ls -Z 
ls --lcontext


Examples:


[root@osboxes rc.d]# cd /etc/rc.d/

[root@osboxes rc.d]# ls -alt
total 76
drwxr-xr-x. 121 root root 12288 Aug 15 00:41 ..
drwxr-xr-x.   2 root root  4096 Aug  6 07:33 rc0.d
drwxr-xr-x.   2 root root  4096 Aug  6 07:33 rc1.d
drwxr-xr-x.   2 root root  4096 Aug  6 07:33 rc2.d
...

[root@osboxes rc.d]# ls -Z 
drwxr-xr-x. root root system_u:object_r:etc_t:s0       init.d
-rwxr-xr-x. root root system_u:object_r:initrc_exec_t:s0 rc
drwxr-xr-x. root root system_u:object_r:etc_t:s0       rc0.d
drwxr-xr-x. root root system_u:object_r:etc_t:s0       rc1.d
drwxr-xr-x. root root system_u:object_r:etc_t:s0       rc2.d
...

[root@osboxes rc.d]# ls --lcontext 
total 60
drwxr-xr-x. 2 system_u:object_r:etc_t:s0       root root  4096 Aug  6 07:33 init.d
-rwxr-xr-x. 1 system_u:object_r:initrc_exec_t:s0 root root  2617 May 11 20:32 rc
drwxr-xr-x. 2 system_u:object_r:etc_t:s0       root root  4096 Aug  6 07:33 rc0.d
drwxr-xr-x. 2 system_u:object_r:etc_t:s0       root root  4096 Aug  6 07:33 rc1.d
drwxr-xr-x. 2 system_u:object_r:etc_t:s0       root root  4096 Aug  6 07:33 rc2.d
...


Remove SELinux permissions from files

SELinux context remains associated with files regardless whether or not SELinux is enabled.

If you are like me and you find the trailing "dot" annoying in the permissions, the following command will remove the dots (remove the SELinux context).


sudo setfattr -h -x security.selinux 

Examples:


sudo setfattr -h -x security.selinux /home

find /home -print0 |xargs -0 -n 1 sudo setfattr -h -x security.selinux

find /home -exec sudo setfattr -h -x security.selinux {} \;

(comment on this)

Monday, August 8th, 2016
7:06 am - Plot Holes in Suicide Squad
All the things wrong with Suicide Squad

1- Dude blows himself up  - The soldier who Flag tells to detinate the bomb BLOWS HIMSELF UP.     If you order someone to kill themselves, do you really think they are going to just kill themselves.

2- Harley Throw gun to Deadshot who already has gun on his arm.

*- The Enchantress did not teleport.   Instead of teleporting away, The Enchantress makes an illusion of Deadshot's daughter to try and stop him.  She could have just teleported away.

*- Harley could I have just left, and would have.    At the end when amanda waller shows up with the app on her phone and tells them they are going back to prison, Harley's device was disabled.    So Harley could have just left.  There was nothing holding her there.

(comment on this)

Friday, August 5th, 2016
11:01 am - Apple Command + Enter = Middle Mouse Click in Linux
1- Install xdotool.

2- Add a keyboard shortcut for Apple Command + Enter = Middle Mouse Click in Linux


(comment on this)

Tuesday, August 2nd, 2016
1:03 pm - How to grep a Java Stacktrace

If you've ever had to deal with java stacktraces you know that they are a real pain to have to try and grep through.

If you want to just capture the stacktrace that matches a single string you are screwed because it's a multiple line glob.

Here's how to grep a java stacktrace using good old awk.  For this example, I'm using the date stamp (2016) for my delimiter for a new stacktrace stanza.

The string I'm searching for is "Status Code: 404" and notice how it returns just the stacktrace for me that matches that string.


[foo@myhost ~]# awk 'BEGIN{RS="\n2016-";ORS="\n\n2016-";FS=OFS="\n"}/Status Code: 404/' ~tomcat/logs/tomcat.log |more
08-02 16:10:37,581 http-8443-71 ERROR DefaultExceptionHandler:82 - Unhandled exception in exception handler. - XXXXXXXXXX
com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: XXXXXXXXXXX), S3 Extended Request ID: XXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
    at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1191)
    at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1059)
    at sun.reflect.GeneratedMethodAccessor1236.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at net.sourceforge.stripes.controller.DispatcherHelper$6.intercept(DispatcherHelper.java:467)
    at net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:158)
    at net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:155)
    at net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:155)
    at net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:155)
    at net.sourceforge.stripes.controller.BeforeAfterMethodInterceptor.intercept(BeforeAfterMethodInterceptor.java:113)
    at net.sourceforge.stripes.controller.ExecutionContext.proceed(ExecutionContext.java:155)
    at net.sourceforge.stripes.controller.ExecutionContext.wrap(ExecutionContext.java:74)
    at net.sourceforge.stripes.controller.DispatcherHelper.invokeEventHandler(DispatcherHelper.java:465)
    at net.sourceforge.stripes.controller.DispatcherServlet.invokeEventHandler(DispatcherServlet.java:278)
    at net.sourceforge.stripes.controller.DispatcherServlet.service(DispatcherServlet.java:160)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at net.sourceforge.stripes.controller.StripesFilter.doFilter(StripesFilter.java:247)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:646)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:436)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:374)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:302)
    at com.hazelcast.web.WebFilter$RequestWrapper$1.forward(WebFilter.java:428)
    at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite(NormalRewrittenUrl.java:213)
    at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:171)
    at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
    at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
    at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

(comment on this)

Sunday, July 31st, 2016
5:04 pm - Setting up CentOS 6.8 in VirtualBox

Setting up CentOS 6.8 in VirtualBox

1- Install VirtualBox 5.1.2

https://www.virtualbox.org/wiki/Downloads

2- Install VirtualBox 5.1.2 Oracle VM VirtualBox Extension Pack

http://download.virtualbox.org/virtualbox/5.1.2/Oracle_VM_VirtualBox_Extension_Pack-5.1.2-108956.vbox-extpack

3- Download CentOS 6.8 image for VirtualBox

http://www.osboxes.org/centos/

Option A) Uncompress image using  Unarchiver

Download "Unarchiver" and use it to extract the image: http://unarchiver.c3.cx/unarchiver

Option B) Uncompress image using p7zip

Install Xcode (needed for HomeBrew)

And if you don't have Xcode installed... you'll need to install Xcode (smile)

https://developer.apple.com/xcode/downloads/

Install HomeBrew

If you don't have "HomeBrew" installed, you'll need to install HomeBrew.

http://brew.sh/

Install p7zip

If you don't have 7z tools installed you'll need to install them with brew:

brew install p7zip

Uncompress the image

cd ~/Downloads/
7za e CentOS_6.8-VB-32bit.7z

4- Launch the downloaded image inside of VirtualBox

5- Install the VirtualBox Guest Additions into the Guest OS

http://www.if-not-true-then-false.com/2010/install-virtualbox-guest-additions-on-fedora-centos-red-hat-rhel/


Note: you will need to be root.  If you "sudo su -" fails, then "ssh 0 -l root"

6- Reboot

sudo reboot



------------ Once you have the above working -----------------

7- Switch to Bridged Adapter

After you get everything up and running switch to Bridged Adapter, and restart your VM.

(This way you can ssh from the host to the guest via the guest IP address).

(comment on this)

Monday, July 25th, 2016
3:27 pm - Installed gnu find under macosx via brew

# brew install coreutils findutils gnu-tar gnu-sed gawk gnutls gnu-indent gnu-getopt

Here's a better way:


brew install binutils
brew install diffutils
brew install ed --with-default-names
brew install findutils --with-default-names
brew install gawk
brew install gnu-indent --with-default-names
brew install gnu-sed --with-default-names
brew install gnu-tar --with-default-names
brew install gnu-which --with-default-names
brew install gnutls
brew install grep --with-default-names
brew install gzip
brew install screen
brew install watch
brew install wdiff --with-gettext
brew install wget


https://www.topbug.net/blog/2013/04/14/install-and-use-gnu-command-line-tools-in-mac-os-x/

(comment on this)

Sunday, June 26th, 2016
11:26 am - Facebook has a silent homophobic policy which they enforce against gay men.
Facebook has a silent homophobic policy which they enforce against gay men.

I’ll give you 2 examples of photos that I posted on Facebook and each time I was placed in Facebook Jail for “nudity.” Please keep in mind, I have not, nor would I ever post nudity on Facebook. And you’ll see from the photos that there’s no nudity in any of these pictures. And yet, each one was removed by Facebook and my account suspended, also known as “Facebook Jail.”

Since I can’t post this on facebook, I'll post it here on my livejournal account.
1- http://jackal.livejournal.com/22065... - Second time placed in Facebook Jail for posting a photo that contained NO NUDITY.  But I was Facebook jailed for "nudity."

2- http://jackal.livejournal.com/22023...  - This one’s confusing. The image was reviewed and cleared as it “doesn't violate this community standard.” And then I reposted it cropped, and the repost got me put in Facebook Jail. Even though it had already been cleared as it “doesn't violate this community standard.”

Both times my account was suspended and I was placed in Facebook Jail for "nudity" and as you can see from the photos there was no nudity in these photos.

(2 comments | comment on this)

Saturday, June 25th, 2016
7:00 pm - Rules for Playing Juggernaut(s): A Solitary Game for Magic

https://www.youtube.com/watch?v=0zPWENsXHsg&feature=youtu.be

Rules for Playing Juggernaut(s):

1- Both players start with 20 life points

2- There's only 1 Library and only 1 Graveyard (yours).  So if anything says to put a card on top or bottom of opponents library, it goes on the top or bottom of your library instead.  And if you kill an opponents creature it goes into your graveyard.

3- The opponent does not start playing until turn #3.

4- You go first.  :)

5- When an opponent summons a Juggernaut it is a 5/3 Artifact Creature with "Juggernaut attacks each turn if able. Juggernaut can't be blocked by Walls." and is denoted by taking the top card off your library and placing it face down on the opponent's side.  Juggernauts in this game have Haste.

6- On your opponent's turn, you role a d6 a take the following actions:


  1. Do nothing

  2. Opponent Summons a Juggernaut

  3. Discard your hand (yes, your whole hand)

  4. Opponent Summons a Juggernaut

  5. Opponent forces you to sacrifice a creature.   Then roll again.

  6. Opponent Summons a Juggernaut

(comment on this)

Friday, May 13th, 2016
3:39 pm - Back in Facebook Jail... again.

This has now happened to me TWICE.

I was just suspended from Facebook AGAIN.  And again, it's because someone reported one of my photos. A photo which contains NO NUDITY, and yet Facbeook removed it for "Nudity" and I'm suspended from Facebook again.

NOTICE HOW THIS PHOTO CONTAINS NO NUDITY!!!!!




And yet I was suspended from Facebook for this.

(comment on this)

Monday, May 2nd, 2016
2:38 pm - How to get crons to work with EDT vs EST on a Ubuntu
Since Ubuntu does not recognize the CRON_TZ varaible in cron, I wrote this little test to make it so this cron will always run at the same time everyday at 7:01am.

#This crons runs at 7:01am every day regardless of EDT or EST.
1 11 * * * root test `TZ=America/New_York date +"\%Z"` = "EDT" && my_script_here
1 12 * * * root test `TZ=America/New_York date +"\%Z"` = "EST" && my_script_here


(comment on this)

Thursday, April 14th, 2016
11:51 am - How to run crons that are unaffected by EDT/EST time change.

The problem with running crons with UDT is that if you want them to run at the same time relative to East Coast time, you'll have to change all your crons when EST<->EDT time switches. And that's a real pain.

For example, I have a cron that runs at 7 AM every morning. I convert that to UTC and put in my cron as 11 UTC.
Then the clocks change, and guess what? 11 UTC is no longer 7am. It's now 6am instead. And I dont' want my cron to run at 6am. I want it to run at 7am. So I have to go change all my crons to change the hour when EDT<->EST happens.

I found a way around that, that allows me to keep my systems in UTC AND not be affected by EDT<->EST time change.

For example, if you want a cron to run at 11:43AM every day no matter what, he's how:

Example 1 (date prints in UTC):


[root@XXXXXX]# cat /etc/cron.d/cron_test 
# this should run at 11:43 am East Coast time regardless of EDT/EST
CRON_TZ=America/New_York
43 11 * * * root date >> /var/log/cron-test.out

Example 2 (force date to print in EDT/EST time):

[root@XXXXXX]# cat /etc/cron.d/cron_test 
# this should run at 11:43 am East Coast time regardless of EDT/EST
CRON_TZ=America/New_York
43 11 * * * root TZ="America/New_York" date >> /var/log/cron-test.out

(comment on this)

Tuesday, March 29th, 2016
10:54 pm - Google is BLOCKING DNS for www.thepiratebay.se
This is interesting. Google is BLOCKING DNS for www.thepiratebay.se

$ nslookup www.thepriatebay.se 8.8.8.8
Server:        8.8.8.8
Address:    8.8.8.8#53

** server can't find www.thepriatebay.se: NXDOMAIN

(comment on this)

Sunday, March 20th, 2016
4:49 pm - How to run processes in parallel in bash and use pipelining.
This one's even better than the last.

The last one would wait until the last process in the batch was finished before starting a new batch.

This one starts a new process as soon as one exits up until the limit of the max allowed.

https://raw.githubusercontent.com/jackal242/brads_scripts/master/parallel_forked_processes.sh

(comment on this)

Friday, March 18th, 2016
3:41 pm - How to do Rate Limiting on While Looped Backgrounded/Forked Processes in Bash For Parallelism
MAX_BATCH_FILES_COUNT=5    # how many log files to work at the same time
open_files=0
for i in `find /x/FILES -name "*gz" |sort`; do
        let "open_files++"  # increment open files count
        # echo $i
        print_errors_count $i &  # this function does all my heavy work, and is forked into background
        pids[${open_files}]=$!;
        if [[ "$open_files" -ge "$MAX_BATCH_FILES_COUNT" ]]; then
                echo "WAITING NOW.  open files=$open_files"
                open_files=0   # Reset back to zero
                for pid in ${pids[*]}; do
                        wait $pid;   # this will wait for them all to exit before starting the next batch
                done;
        else
                echo "open_files=$open_files"
        fi
done

(comment on this)

Thursday, February 11th, 2016
2:25 pm - bash script for telling you when your Kerberos password is going to expire (freeipa)
#!/bin/bash

LDAPSERVER=freeipa.XXXXXX.XXXXXXX.com

if [[ "$USER" == "root" ]];then
    exit
fi

LDAPDATA=`ldapsearch -N -Y GSSAPI -h $LDAPSERVER -b "cn=users,cn=accounts,dc=XXXXX,dc=XXXXX,dc=com" uid=$USER krbPasswordExpiration 2>&1`
if [[ $? -ne 0 ]]; then
    echo "ldapsearch command failed. Do you have valid Kerberos tickets for $USER?"
    exit
fi
EXPIRE_DATE_TIME=`/bin/echo $LDAPDATA |grep "krbPasswordExpiration:"`
if [[ $? -ne 0 ]]; then
    echo "ldapsearch command failed. No krbPasswordExpiration data returned for $USER"
    exit
fi
EXPIRE_DATE_TIME=`/bin/echo $EXPIRE_DATE_TIME |sed -r "s/.*krbPasswordExpiration: ([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9])([0-9][0-9][0-9][0-9]).*/\1 \2/"  `
EXPIRE_DATE_EPOCH=`date --date "$EXPIRE_DATE_TIME" +%s`
NOW_DATE_EPOCH=`date +%s`
DATE_DIFF=`expr $EXPIRE_DATE_EPOCH - $NOW_DATE_EPOCH`
DAYS=`expr $DATE_DIFF / 86400`
echo "Your Kerberos Password expires in $DAYS days ($USER, `date -d @${EXPIRE_DATE_EPOCH}`)"




(comment on this)

Tuesday, February 9th, 2016
1:54 pm - undocumented flag in mtr
MTR has a --first-ttl (or -f) option that does NOT show up in the man page for mtr.

The default for this value is 1.

/usr/sbin/mtr --first-ttl 5 --report-wide --report --report-cycles 10 www.google.com

(comment on this)

> previous 20 entries
> top of page
LiveJournal.com