manageProcess + redis 实现 ppt 转 pdf 服务

最近接到一个把 ppt 转成 pdf 的需求,和客户端约定 ppt 最大需要50M,用户上传ppt 文件到服务器,服务器经过一系列的处理搞定转化工作,最后客户端在将成品的pdf文件下载下来,写个博客记录下。

首先是技术选型:服务端有个 libreoffice 插件,可以完成 ppt 转 pdf 的功能,和客户端简单的测试了下,发现效果不错,来两张转化的效果图:

可以看到效果还是相当不错的,源文件是我从网上随便 down 的,具体链接可以参考我下面的附录;

接口设计:上传 upload 接口上传 ppt 文件;轮询 process 接口查询该文件处理进度;download 接口下载转换成功的 pdf 文件;

接下来是架构实现:客户端上传到服务端,文件上传嘛!服务端上传文件到客户端,表单上传搞定。框架现有的 upload 类轮子,我这吭哧吭哧改了一顿代码之后,自测通过,pass!(这里给自己留了个大坑,留待下一个模块慢慢说)

服务端处理,考虑到 php 处理一个服务是 php-fpm 进程搞定的,直接上手简直爆炸,把一个最大可能50M,最小也得好几M的文件放在 php-fpm 里面处理,一来可能导致当前的 php-fpm 进程阻塞,二来严重阻碍其他 request 请求的顺利执行,三来我们都晓得计算开启 php-fpm 进程数量很重要一个参考因素是 php-fpm 进程占服务器内存20-30m来算的;所以直接使用php-fpm进程搞定就被我们给否了,但是这个服务又不需要太多的并发流量;

考虑再三,使用redis list 实现一个队列,manageProcess 快速实现一个多进程任务,抢占式的 pop redis 队列,上传时创建全局 uuid 并使用 redis 记录处理的进度;

按照事先的接口设计实现代码,服务器设置下跨域,本地一个 form 表单上传完美搞定。

第二天,与客户端联调,本想在前端小姐姐面前秀一波,没想到第一步就卡了,upload 请求收不到,什么情况,查下nginx,发现请求来了,但是我没有正确反馈,wtf???

细问下了解到原来客户端用的是nodejs,根本没有 form 表单一说(呼应开头的那个坑),好吧找到问题,那就现场撸代码,php 有个神奇的 php://input 自动获得http body 数据流,我直接强行读 php://input 然后写文件不就搞定了,perfect 解决!

还有其他字体的坑,不是本文核心就不一一列举了,贴下核心代码:

while (true) {
            $this->flushTimestamp($pid);
            $this->nowUuid = null;
            $fileInfo = $redis->rpop(self::prefix);
            if(!$fileInfo) {
                sleep(1);
                continue;
            }
            $output = '';
            $uuid = $fileInfo['uuid'];
            $this->nowUuid = $uuid;
            $savePath = $fileInfo['savePath'];
            $ext = $fileInfo['ext'];
            $filename = $savePath . $uuid . ".$ext";
            $res = [
                'status' => 2,
                'uuid' => $uuid,
                'msg' => 'wait file sync'
            ];
            $redis->set(self::prefix.'_'.$uuid,$res);
            if(!file_exists($filename)) {
                // 等待 NAS 服务同步数据
                $this->flushTimestamp($pid);
                sleep(3);
                if(!file_exists($filename)) {
                    $res = [
                        'status' => -1,
                        'uuid' => $uuid,
                        'msg' => 'file not found,please try again!'
                    ];
                    $redis->set(self::prefix.'_'.$uuid,$res);
                    $fileInfo['__REASON__'] = 'file not exist';
                    Fend_Log::write($fileInfo,$this->config['logName']);
                    continue;
                }
            }
            if(md5_file($filename) != $fileInfo['filemd5']) {
                $res = [
                    'status' => -2,
                    'uuid' => $uuid,
                    'msg' => 'file damage!'
                ];
                $redis->set(self::prefix.'_'.$uuid,$res);
                $fileInfo['__REASON__'] = 'file damage in nas!';
                Fend_Log::write($fileInfo,$this->config['logName']);
                continue;
            }
            $res = [
                'status' => 2,
                'uuid' => $uuid,
                'msg' => 'process transforming...'
            ];
            $redis->set(self::prefix.'_'.$uuid,$res);
            exec("/usr/bin/libreoffice  --invisible --convert-to pdf --outdir $savePath ".$filename,$output);
            $res = [
                'status' => 1,
                'execOut' => $output,
                'msg' => 'transform complete',
                'path' => $savePath,
                'name' => $uuid. ".$ext",
                'uuid' => $uuid,
                'ext' => $ext,
                'fileName' => $filename
            ];
            $redis->set(self::prefix.'_'.$uuid,$res);
        }

附录:

1、pptx 模板文件:http://blog.woai662.net/wp-content/uploads/2018/01/job.pptx

2、转化完成的pdf连接:http://blog.woai662.net/wp-content/uploads/2018/01/job.pdf

3、参考的文章连接:http://blog.csdn.net/ljihe/article/details/77250206

 

“manageProcess + redis 实现 ppt 转 pdf 服务”的9,115个回复

  1. Working as a frеelance paralegal has elements in its
    favor, and factors which might be detrimеntal to some people.
    If a ԝaү of adventure and pⅼeasure in your work life is what wоᥙld suit you
    one of the best, freelancing might be an aѡesome choice for you!

  2. You mean like after we sing prаise songs in Cһurch?? Larrty requested and daⅾdy nodded.
    ?Effectively I could make up a worship song.?

    Ꮪo Larry jumped to his toes and started to make up a musіc to a reаlly bad tune.

    ?Jesus iѕ so cool. Its fun being with God. Нes the
    funnest God аnybody might have.? Ladry sang very badly so Lеe hɑd ρut his hand over his ears.

  3. Hi there, i read your blog from time to time and i own a similar one and i was just wondering if you get a lot
    of spam responses? If so how do you reduce it, any
    plugin or anything you can recommend? I get so much lately
    it’s driving me insane so any support is very much appreciated.

  4. My coder is trying to convince me to move to .net from PHP.
    I have always disliked the idea because of the expenses. But he’s tryiong none the less.
    I’ve been using Movable-type on several websites for about a
    year and am anxious about switching to another platform. I
    have heard very good things about blogengine.net. Is there a
    way I can import all my wordpress posts into it? Any help would be greatly appreciated! https://www.transifex.com/user/profile/TrevorGriggs/

  5. I think what you typed was very logical. But, what about this?
    what if you added a little content? I ain’t saying
    your content isn’t good., however what if you added something to
    possibly get folk’s attention? I mean manageProcess + redis 实现 ppt 转 pdf 服务
    – weizengreal is kinda boring. You might peek at
    Yahoo’s home page and see how they create news headlines to get viewers interested.
    You might try adding a video or a picture or two to get people interested about
    everything’ve got to say. In my opinion, it could make your website a little bit more interesting. http://bukpravda.cv.ua/news/chernovtsy/item/41096

  6. hey there and thank you for your information –
    I’ve definitely picked up anything new from right here. I did however expertise some technical issues using this website, since I experienced to reload the web site many
    times previous to I could get it to load correctly. I had been wondering if your web hosting
    is OK? Not that I am complaining, but slow loading instances times will
    often affect your placement in google and could damage your high-quality score if advertising and marketing with Adwords.

    Anyway I’m adding this RSS to my e-mail and can look out for much
    more of your respective fascinating content. Make sure you update this again very soon. https://id.arduino.cc/DaydayGurlitt

  7. Needed to send you that bit of remark just to say thanks once again on your exceptional tactics you have featured on this site.

    This has been seriously generous of you to give publicly what most
    people could have offered as an e-book to make some profit for their
    own end, principally now that you might have tried it if you ever wanted.
    The inspiring ideas also acted to be a easy way to
    comprehend most people have similar keenness really like my personal own to know many more in regard
    to this matter. I’m sure there are some more enjoyable situations ahead for folks who take a look at your blog post. https://www.gifts4promo.co.uk

  8. Hey I am so grateful I found your webpage, I really found you
    by accident, while I was looking on Yahoo for
    something else, Anyhow I am here now and would
    just like to say thank you for a remarkable post and a all round exciting blog (I also
    love the theme/design), I don’t have time to look over it
    all at the moment but I have saved it and also added your
    RSS feeds, so when I have time I will be back to read a great deal more, Please do keep up the great job. http://bestworkboots1.jigsy.com/entries/general/Safety-Boots–Maintain-Your-Feet-Protected

  9. After going over a few of the blog posts on your website, I honestly like your technique of blogging.
    I book-marked it to my bookmark site list and will be checking back soon. Please visit
    my website too and let me know your opinion.

  10. Hi there great blog! Does running a blog like this take a
    massive amount work? I’ve virtually no understanding of computer programming but I
    was hoping to start my own blog in the near future.
    Anyways, if you have any ideas or tips for new blog owners please share.
    I understand this is off topic however I just wanted to ask.
    Kudos!