데몬으로 python script 를 실행할때 직접 daemonizing 코드를 구형하여 만들었는데

python-daemon 이라는 패키지를 사용해보니 cmdline 에서 start/stop/restart 도 지원하고 filelock 도 지원한다.

쓸만해보여서 정리해둠 


wget https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tgz
tar zxf Python-2.7.12.tgz
cd Python-2.7.12
./configure --prefix=/home/apps/python2.7 --enable-unicode=ucs4 --enable-shared
make
make install
wget --no-check-certificate https://pypi.python.org/packages/25/4e/1b16cfe90856235a13872a6641278c862e4143887d11a12ac4905081197f/setuptools-28.8.0.tar.gz
tar zxf setuptools-28.8.0.tar.gz
cd setuptools-28.8.0
python2.7 setup.py build
python2.7 setup.py install
wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py
python2.7 get-pip.py
pip install python-daemon
 
mkdir /daemon_sample
cd /daemon_sample
mkdir  bin lib log run
cd bin
vi daemon_sample.py
--------------------------------------------------------------------------------------------
#!/home/apps/python2.7/bin/python
# -*- coding: UTF-8 -*-
import time
from daemon.runner import DaemonRunner
class TestDaemon(object):
    def __init__(self):
        self.stdin_path = '/dev/null'
        self.stdout_path = '/dev/tty'
        self.stderr_path = '/dev/tty'
        self.pidfile_timeout = 0
        self.pidfile_path="/usr/mgmt/apm_package_manager/run/TestDaemon.pid"
    def run(self):
        while 1:
            time.sleep(3)
DaemonRunner(TestDaemon()).do_action()
--------------------------------------------------------------------------------------------
chmod 700 daemon_sample.py
./daemon_sample.py
usage: test.py start|stop|restart
 
 
./daemon_sample.py start
started with pid 24751
 
 
./daemon_sample.py start
  File "./test.py", line 19, in <module>
    DaemonRunner(TestDaemon()).do_action()
  File "/home/apps/python2.7/lib/python2.7/site-packages/daemon/runner.py", line 274, in do_action
    func(self)
  File "/home/apps/python2.7/lib/python2.7/site-packages/daemon/runner.py", line 187, in _start
    raise error
daemon.runner.DaemonRunnerStartFailureError: PID file '/usr/mgmt/apm_package_manager/run/TestDaemon.pid' already locked


'Python' 카테고리의 다른 글

Pika Python AMQP Client Library  (0) 2017.01.31
s3 example  (0) 2016.12.28
Threading  (0) 2016.12.22
pidlockfile.py for windows  (0) 2016.12.19
cygwin + ssh + rsync  (0) 2016.12.10

python 에서 서브쓰레드를 정상종료해보자 

시그널은 메인쓰레드만 받을수 있다.

메인쓰레드에서 시그널을 받아 서브쓰레드에 이벤트로 플래그를 전달한다.


#!/usr/local/python2.6/bin/python
#-*- coding: utf-8 -*-
import threading
import time
import logging
import random
import Queue
import signal
import sys
logging.basicConfig(level=logging.DEBUG,
                    format='(%(threadName)-9s) %(message)s',)
 
BUF_SIZE = 10
q = Queue.Queue(BUF_SIZE)
e = threading.Event()
 
class ProducerThread(threading.Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs=None, verbose=None):
        super(ProducerThread,self).__init__()
        self.target = target
        self.name = name
    def run(self):
        while not e.is_set():
            if not q.full():
                item = random.randint(1,10)
                q.put(item)
                logging.debug('Putting ' + str(item)
                              ' : ' + str(q.qsize()) + ' items in queue')
        logging.debug("end")
        return
 
class ConsumerThread(threading.Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs=None, verbose=None):
        super(ConsumerThread,self).__init__()
        self.target = target
        self.name = name
    def run(self):
        while not e.is_set():
            if not q.empty():
                item = q.get()
                logging.debug('Getting ' + str(item)
                              ' : ' + str(q.qsize()) + ' items in queue')
        logging.debug("end")
        return
 
if __name__ == '__main__':
    p = ProducerThread(name='producer')
    c = ConsumerThread(name='consumer')
    p.setDaemon(True)
    c.setDaemon(True)
    p.start()
    c.start()
    try:
        while p.is_alive() and c.is_alive():
            time.sleep(1)
    except KeyboardInterrupt:
        e.set()
 
    p.join()
    c.join()


kill 커맨드로 2번 시그널을 메인쓰레드에 전달한다.

end 구문까지 정상적으로 처리가 되고 종료가 된다.


]# ps -ef | grep tqt.py
root     23927 13186 99 11:55 pts/0    00:00:10 /usr/local/python2.6/bin/python ./tqt.py
 
]# kill -2 23927
 
(producer ) Putting 1 : 1 items in queue
(consumer ) Getting 1 : 0 items in queue
(producer ) Putting 7 : 1 items in queue
(consumer ) Getting 7 : 0 items in queue
(producer ) Putting 5 : 1 items in queue
(consumer ) Getting 5 : 0 items in queue
(producer ) Putting 2 : 1 items in queue
(consumer ) Getting 2 : 0 items in queue
(producer ) Putting 1 : 1 items in queue
(consumer ) Getting 1 : 0 items in queue
(producer ) Putting 1 : 1 items in queue
(consumer ) Getting 1 : 0 items in queue
(producer ) Putting 2 : 1 items in queue
(consumer ) Getting 2 : 0 items in queue
(producer ) Putting 2 : 1 items in queue
(consumer ) Getting 2 : 0 items in queue
(producer ) Putting 2 : 1 items in queue
(consumer ) Getting 2 : 0 items in queue
(producer ) Putting 10 : 1 items in queue
(consumer ) end
(producer ) end


'Python' 카테고리의 다른 글

s3 example  (0) 2016.12.28
daemonizing  (0) 2016.12.22
pidlockfile.py for windows  (0) 2016.12.19
cygwin + ssh + rsync  (0) 2016.12.10
remove ^M(Carriage Return)  (0) 2016.11.30

윈도우 가상서버 디스크이미지(vhd) 파일을 복사해서 다른 Hyper-V  서비스에서 디스크 추가를할때


permission 에러가 발생하는 경우가 있다.


vm  SID 를 vhd 파일에 추가해줘야한다.


디렉토리는 Virtual Machines 그룹이 추가되어 있어야 한다.


https://support.microsoft.com/en-us/kb/2249906



import os

user_id = "vm001"

drv = os.path.join("C:\\", "vm_disks")

for file in os.listdir(os.path.join(drv, user_id, "Virtual Machines")):

    if file.endswith(".xml"):

        vm_id = os.path.splitext(file)[0]

        disk_dir = os.path.join(drv, user_id, "Virtual Hard Disks")

        for disk_file in os.listdir(disk_dir):

            if disk_file.endswith(".vhd"):

                cmd = "icacls \"%s\" /grant \"NT VIRTUAL MACHINE\\%s\":(F)" % (os.path.join(disk_dir, disk_file), vm_id)

                print cmd

                res = os.popen(cmd).read()

                print res



기존에 있던 lockfile.pidlockfile 모듈을 참조하여 pid 부분을 수정


import os

import errno

from win32com.client import GetObject


class PIDLockError(Exception):

    pass


class AlreadyLocked(PIDLockError):

    def __init__(self, path):

        self.path = path

        

    def __str__(self):

        return "%s is already locked" % self.path


class LockFailed(PIDLockError):

    def __init__(self, path):

        self.path = path


    def __str__(self):

        return "failed to create %s" % self.path


class PIDLockFile(object):

    def __init__(self, path):

        self.path = path


    def read_pid(self):

        return self.read_pid_from_pidfile()


    def is_locked(self):

        return os.path.exists(self.path)


    def i_am_locking(self):

        running_process = []

        wmi = GetObject('winmgmts:')

        processes = wmi.InstancesOf('win32_Process')

        for pro in processes:

            running_process.append(pro.Properties_('ProcessId').value)

            

        if self.is_locked():

            if self.read_pid() in running_process:

                return True

        return False


    def acquire(self):

        if not self.i_am_locking():

            self.break_lock()

            

        try:

            self.write_pid_to_pidfile()

        except OSError, exc:

            if exc.errno == errno.EEXIST:


                raise AlreadyLocked(self.path)

            else:

                raise LockFailed(self.path)

        else:

            return


    def release(self):

        self.remove_existing_pidfile()


    def break_lock(self):

        self.remove_existing_pidfile()

        

    def read_pid_from_pidfile(self):

        pid = None

        try:

            pidfile = open(self.path, 'r')

        except IOError:

            pass

        else:

            line = pidfile.readline().strip()

            try:

                pid = int(line)

            except ValueError:

                pass

            pidfile.close()

        return pid


    def write_pid_to_pidfile(self):

        pidfile_fd = os.open(self.path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)

        pidfile = os.fdopen(pidfile_fd, 'w')

        pid = os.getpid()

        pidfile.write("%s\n" % pid)

        pidfile.close()        


    def remove_existing_pidfile(self):

        if self.is_locked(): os.remove(self.path) 



# example

import os

import traceback

import pidlockfile import PIDLockFile, PIDLockError


def main():

    pid_file = os.path.join("C:\\", "run", "run.pid")

    plock = PIDLockFile(pid_file)

    try:

        plock.acquire()

        print "hellow" 

        plock.release()

        

    except PIDLockError, exc:

        print exc

    except Exception, exc:

        print traceback.format_exc()

        

if __name__ == "__main__":

    main()



'Python' 카테고리의 다른 글

daemonizing  (0) 2016.12.22
Threading  (0) 2016.12.22
cygwin + ssh + rsync  (0) 2016.12.10
remove ^M(Carriage Return)  (0) 2016.11.30
sqlalchemy  (0) 2016.11.28

1. pexpect 사용해보려고 하였으니 윈도우에서 사용이 불가

python27, python53 설치 후 pexpect 설치하여 테스트 해보았으나 동작하지 않음

대안으로 winexpect 라는것이 있는데 오픈소스를 더이상 관리하지 않고 종료된 상태


2. tcl.exe 를 이용하여 윈도우에서 expect 기능을 사용할수 있으나 

python 에서 호출할경우에 tcl 스크립트 호출해야하는 python -> tcl -> output 구조가 되서 에러처리에 문제가 있어보임


1/2 번 모두 비밀번호 입력을 자동화 하려고 하것인데

생각해보니 서버단에서 개인키를 핸들링하면 키로 인증하여 윈도우서버간에 파일전송을 할수가 있음

(파일전송후에는 키를 삭제하면됨)


src나 dst 경로에 공백이 들어갈 경우에 Administrator@test.com:'/cygdrive/e/path\ to\ directory' 와 같이 처리해주면됨


import os
cmd = "rsync -av -e \"ssh -o \"StrictHostKeyChecking=no\" -i /cygdrive/c/id_rsa -p 2222\" "
cmd += "Administrator@test.com:'/cygdrive/e/path\ to\ directory' /cygdrive/f"
os.system(cmd)


'Python' 카테고리의 다른 글

Threading  (0) 2016.12.22
pidlockfile.py for windows  (0) 2016.12.19
remove ^M(Carriage Return)  (0) 2016.11.30
sqlalchemy  (0) 2016.11.28
pexpect.pxssh  (0) 2016.11.24


mail block 여부를 확인해야하는데 telnet 연결을 하면 block이 되더라도 연결은 성공이기 때문에 $? 값이 0 으로 참이다 ㅡㅜ 
($? will contain the exit status of the last command)
그래서 메시지로 확인을 해야하는 경우가 있어 구글링해봄


# exec 3<>/dev/tcp/mx4.naver.com/25

# RESPONSE="`cat <&3`"

# echo "Response is: $RESPONSE"

Response is: 220 mx.naver.com ESMTP uhOoOzccQ6eHbxMheoMwfA - nsmtp

451 4.4.2 Timeout - closing connection uhOoOzccQ6eHbxMheoMwfA - nsmtp


# exec 3<>/dev/tcp/mx4.naver.com/25

# RESPONSE="`cat <&3`"

# echo "Response is: $RESPONSE"

Response is: 421 4.3.2 Your ip blocked from this server O8n3JcaPR-SkE+jwsts6GA - nsmtp 



if [[ "$RESPONS" =~ "blocked" ]]; then

    echo "blocked"

else

    echo "ok"

fi



http://code.activestate.com/recipes/286229-remove-control-character-m-from-opened-html-files/

http://blogger.pe.kr/170

>>> import curses.ascii
>>> ascii.ascii('^V^M')
'\r'
>>> string.replace( str, '\r', '' )


'Python' 카테고리의 다른 글

pidlockfile.py for windows  (0) 2016.12.19
cygwin + ssh + rsync  (0) 2016.12.10
sqlalchemy  (0) 2016.11.28
pexpect.pxssh  (0) 2016.11.24
PEP8  (0) 2016.11.24

sqlalchemy + python-daemon


]# /home/python2.7/bin/pip install SQLAlchemy

]# /home/python2.7/bin/pip install mysql-python

]# mkdir /home/package_manager

]# cd /home/package_manager

]# mkdir bin lib log run

]# cd bin

]# vi package_manager.py


#!/home/python2.7/bin/python

# -*- coding: UTF-8 -*-


import sys

import time

from daemon.runner import DaemonRunner

sys.path.append("/home/package_manager/lib")


class App(object):

    def __init__(self):

        self.stdin_path = "/dev/null"

        self.stdout_path = "/dev/tty"

        self.stderr_path = "/dev/tty"

        self.pidfile_timeout = 0

        self.pidfile_path = "/home/package_manager/run/app.pid"


    def run(self):

        from database import session

        from models import APMDistributionQueue

        while 1:

            tasks = session.query(APMDistributionQueue).all()

            for task in tasks:

                print task.server_info.hostn


            time.sleep(10)


def main():

    DaemonRunner(App()).do_action()


if __name__ == "__main__":

    main() 


]# cd lib

]# vi database.py


 # -*- coding: UTF-8 -*-

from sqlalchemy import create_engine

from sqlalchemy.orm import scoped_session, sessionmaker

from sqlalchemy.ext.declarative import declarative_base


engine = create_engine('mysql+mysqldb://아이디:비밀번호@localhost/manager_new', convert_unicode=True)

Base = declarative_base()

session = scoped_session(sessionmaker(autocommit=False, autoflush=False, bind=engine))


]# vi models.py


# -*- coding: UTF-8 -*-

from sqlalchemy.sql import func

from sqlalchemy import ForeignKey, Column, Integer, String, DateTime, Table

from sqlalchemy.orm import relationship, backref

from database import Base, engine


class ServerInfo(Base):

       __table__ = Table('info', Base.metadata, autoload=True, autoload_with=engine)


class APMPackage(Base):

    __tablename__ = 'apm_package'

    id = Column(Integer, primary_key=True)

    name = Column(String(100), unique=True)

    packaging_file = Column(String(200))

    unpackaging_path = Column(String(200))

    queue = relationship("APMDistributionQueue", backref=backref('package'), cascade='all, delete, delete-orphan')


    def __init__(self, name, file, path):

        self.name = name

        self.packaging_file = file

        self.unpackaging_path = path


    def __repr__(self):

        return "%s %s %s %s" % (self.id, self.name, self.packaging_file, self.unpackaging_path)


class APMDistributionQueue(Base):

    __tablename__ = 'apm_distribution_queue'

    id = Column(Integer, primary_key=True)

    mode = Column(String(20), nullable=False)

    server_id = Column(Integer, ForeignKey(ServerInfo.idx), nullable=False)

    package_id = Column(Integer, ForeignKey(APMPackage.id), nullable=False)

    status = Column(Integer, default=1, nullable=False)

    reg_date = Column(DateTime(timezone=True), default=func.now(), nullable=False)

    server_info = relationship("ServerInfo", backref=backref('queue'))


    def __init__(self, mode, sid, pid):

        self.mode = mode

        self.server_id = sid

        self.package_id = pid


    def __repr__(self):

        return "%s %s %s %s" % (self.id, self.mode, self.server_id, self.package_id) 


]# cd ../bin

]# package_manager.py start


프로세스가 fork 되면서 sqlalchemy 의 connection pool과의 연결이 끊기는 문제가 발생해서 한참을 고생했다.

해결방법은 run 메소드 안에서 database.py 를 import 하거나 engine 생성시에 pool_recycle 값을 0으로 설정하면 된다.


새로 만든 테이블과 기존에 있던 테이블을 같이 사용하기 위해 declarative_base, automap_base 중 어떤 맵핑클래스를 

사용하는지에따라 구현하는부분이 달라지는데 테스트코드를 만들어봐야 이해할듯하다.


http://programtalk.com/python-examples/daemon.pidfile.PIDLockFile/

 

 


'Python' 카테고리의 다른 글

pidlockfile.py for windows  (0) 2016.12.19
cygwin + ssh + rsync  (0) 2016.12.10
remove ^M(Carriage Return)  (0) 2016.11.30
pexpect.pxssh  (0) 2016.11.24
PEP8  (0) 2016.11.24

http://pexpect.readthedocs.io/en/stable/api/pxssh.html

'Python' 카테고리의 다른 글

pidlockfile.py for windows  (0) 2016.12.19
cygwin + ssh + rsync  (0) 2016.12.10
remove ^M(Carriage Return)  (0) 2016.11.30
sqlalchemy  (0) 2016.11.28
PEP8  (0) 2016.11.24

+ Recent posts