Skip to content

Commit

Permalink
RecordApp.Stop功能优化,demo添加提升采样率
Browse files Browse the repository at this point in the history
  • Loading branch information
xiangyuecn committed Jan 10, 2020
1 parent e57a289 commit 4cfc685
Show file tree
Hide file tree
Showing 11 changed files with 308 additions and 16 deletions.
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@ $.ajax({
5. [【Demo库】【文件合并】-wav多个片段文件合并](https://xiangyuecn.github.io/Recorder/assets/工具-代码运行和静态分发Runtime.html?jsname=lib.merge.wav_merge)
6. [【教程】实时多路音频混音](https://xiangyuecn.github.io/Recorder/assets/工具-代码运行和静态分发Runtime.html?jsname=teach.realtime.mix_multiple)
7. [【教程】变速变调音频转换](https://xiangyuecn.github.io/Recorder/assets/工具-代码运行和静态分发Runtime.html?jsname=teach.sonic.transform)
8. [【Demo库】PCM采样率提升](https://xiangyuecn.github.io/Recorder/assets/工具-代码运行和静态分发Runtime.html?jsname=lib.samplerate.raise)



Expand Down Expand Up @@ -454,6 +455,8 @@ function transformOgg(pcmData){
### 【静态方法】Recorder.SampleData(pcmDatas,pcmSampleRate,newSampleRate,prevChunkInfo,option)
对pcm数据的采样率进行转换,配合mock方法使用效果更佳,比如实时转换成小片段语音文件。

注意:本方法只会将高采样率的pcm转成低采样率的pcm,当newSampleRate>pcmSampleRate想转成更高采样率的pcm时,本方法将不会进行转换处理(由低的采样率转成高的采样率没有存在的意义);在特殊场合下如果确实需要提升采样率,比如8k必须转成16k,可参考[【Demo库】PCM采样率提升](https://xiangyuecn.github.io/Recorder/assets/工具-代码运行和静态分发Runtime.html?jsname=lib.samplerate.raise)自行编写代码转换一下即可。

`pcmDatas`: [[Int16,...]] pcm片段列表,二维数组

`pcmSampleRate`:48000 pcm数据的采样率
Expand Down
8 changes: 5 additions & 3 deletions app-support-sample/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@

## 【IOS】Hybrid App测试

[demo_ios](https://github.com/xiangyuecn/Recorder/tree/master/app-support-sample/demo_ios)目录内包含IOS App测试源码,和核心文件 [RecordAppJsBridge.swift](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_ios/recorder/RecordAppJsBridge.swift) ;clone后用`xcode`打开后编译运行(没有Mac OS? [装个黑苹果](https://www.jianshu.com/p/cbde4ec9f742) )。本demo为swift代码,兼容IOS 9.0+,已测试IOS 12.3。
[demo_ios](https://github.com/xiangyuecn/Recorder/tree/master/app-support-sample/demo_ios)目录内包含IOS App测试源码,和核心文件 [RecordAppJsBridge.swift](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_ios/recorder/RecordAppJsBridge.swift) ,详细的原生实现、权限配置等请阅读这个目录内的README;clone后用`xcode`打开后编译运行(没有Mac OS? [装个黑苹果](https://www.jianshu.com/p/cbde4ec9f742) )。本demo为swift代码,兼容IOS 9.0+,已测试IOS 12.3。

**xcode测试项目clone后请修改`PRODUCT_BUNDLE_IDENTIFIER`,不然这个测试id被抢来抢去要闲置7天才能被使用,嫌弃苹果公司工程师水准**


## 【Android】Hybrid App测试

[demo_android](https://github.com/xiangyuecn/Recorder/tree/master/app-support-sample/demo_android)目录内包含Android App测试源码,和核心文件 [RecordAppJsBridge.java](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_android/app/src/main/java/com/github/xianyuecn/recorder/RecordAppJsBridge.java) ;目录内 [app-debug.apk.zip](https://xiangyuecn.github.io/Recorder/app-support-sample/demo_android/app-debug.apk.zip) 为打包好的debug包(40kb,删掉.zip后缀),或者clone后自行用`Android Studio`编译打包。本demo为java代码,兼容API Level 15+,已测试Android 9.0。
[demo_android](https://github.com/xiangyuecn/Recorder/tree/master/app-support-sample/demo_android)目录内包含Android App测试源码,和核心文件 [RecordAppJsBridge.java](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_android/app/src/main/java/com/github/xianyuecn/recorder/RecordAppJsBridge.java) ,详细的原生实现、权限配置等请阅读这个目录内的README;目录内 [app-debug.apk.zip](https://xiangyuecn.github.io/Recorder/app-support-sample/demo_android/app-debug.apk.zip) 为打包好的debug包(40kb,删掉.zip后缀),或者clone后自行用`Android Studio`编译打包。本demo为java代码,兼容API Level 15+,已测试Android 9.0。

## 【IOS微信】H5测试
[<img src="https://gitee.com/xiangyuecn/Recorder/raw/master/assets/demo-recordapp.png" width="100px">](https://jiebian.life/web/h5/github/recordapp.aspx) https://jiebian.life/web/h5/github/recordapp.aspx
Expand Down Expand Up @@ -111,7 +111,7 @@ RecordApp.RequestPermission(function(){
}
},function(){
setTimeout(function(){
RecordApp.Stop(function(blob,duration){//到达指定条件停止录音
RecordApp.Stop(function(blob,duration){//到达指定条件停止录音和清理资源
console.log(blob,(window.URL||webkitURL).createObjectURL(blob),"时长:"+duration+"ms");

//已经拿到blob文件对象想干嘛就干嘛:立即播放、上传
Expand Down Expand Up @@ -262,6 +262,8 @@ IOS-Weixin底层会把从微信素材下载过来的原始音频信息存储在s

`fail`: `fn(errMsg)` 录音出错时回调

如果不提供success参数=null时,将不会进行音频编码操作,只进行清理完可能持有的资源后走fail回调。


## 【静态方法】RecordApp.Install(success,fail)
对底层平台进行识别和加载相应的类库进行初始化,`RecordApp.RequestPermission`只是对此方法进行了一次封装,并且多了一个权限请求而已。如果你只想完成功能的加载,并不想调起权限请求,可手动调用此方法。此方法可以反复调用。
Expand Down
9 changes: 9 additions & 0 deletions app-support-sample/demo_android/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,15 @@ java收到js发起的`RecordAppJsBridge.request`请求,解析请求数据参
Android端的录音还算完美,比IOS的轻松很多。


## 需要权限
1. `android.permission.RECORD_AUDIO`
2. `android.permission.MODIFY_AUDIO_SETTINGS`


## 如何接入使用
请阅读[RecordAppJsBridge.java](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_android/app/src/main/java/com/github/xianyuecn/recorder/RecordAppJsBridge.java)文件开头的注释文档,可直接copy此文件到你的项目中使用;支持新开发WebView界面,或对已有的WebView实例升级支持RecordApp。


## 为什么不用UserAgent来识别App环境

通过修改WebView的UA来让H5、服务器判断是不是在App里面运行的,此方法非常简单而且实用。但有一个致命缺陷,当UA数据很敏感的场景下,虽然方便了我方H5、服务器来识别这个App,但也同时也暴露给了任何在此WebView中发起的请求,不可避免的会将我们的标识信息随请求而发送给第三方(虽然可通过额外编程把信息抹掉,但代价太大了)。IOS不动UA基本上别人的服务器几乎不太可能识别出我们的App,Android神一样的把包名添加到了X-Requested-With请求头中,还能不能讲理了。
Expand Down
8 changes: 8 additions & 0 deletions app-support-sample/demo_ios/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ swift收到js发起的prompt弹框请求,解析弹框携带的数据参数,
可能是因为`AVAudioRecorder`存在文件写入缓存的原因,数据并非实时的flush到文件的,因此实时发送给js的数据存在300ms左右的滞后;`AudioQueue``AudioUnit`之类的更强大的工具文章又少,代码又多,本质上是因为不会用,所以就成这样了。


## 需要权限
在plist中配置麦克风的权限声明:`NSMicrophoneUsageDescription`


## 如何接入使用
请阅读[RecordAppJsBridge.swift](https://github.com/xiangyuecn/Recorder/blob/master/app-support-sample/demo_ios/recorder/RecordAppJsBridge.swift)文件开头的注释文档,可直接copy此文件到你的项目中使用;支持新开发WKWebView界面,或对已有的WKWebView实例升级支持RecordApp。


## 为什么不用UserAgent来识别App环境

通过修改WebView的UA来让H5、服务器判断是不是在App里面运行的,此方法非常简单而且实用。但有一个致命缺陷,当UA数据很敏感的场景下,虽然方便了我方H5、服务器来识别这个App,但也同时也暴露给了任何在此WebView中发起的请求,不可避免的会将我们的标识信息随请求而发送给第三方(虽然可通过额外编程把信息抹掉,但代价太大了)。IOS不动UA基本上别人的服务器几乎不太可能识别出我们的App,Android神一样的把包名添加到了X-Requested-With请求头中,还能不能讲理了。
Expand Down
12 changes: 11 additions & 1 deletion app-support-sample/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@

<script>
//兼容环境
var PageLM="2019-11-7 21:49:55";
var PageLM="2020-1-10 11:52:57";

function BuildHtml(html,o,notEncode,loop){return o||(o={}),html=(null==html?"":html+"").replace(/\{(fn:)?(:)?(.+?)\}/gi,function(a,b,c,code){try{var v=eval(b?code:"o."+code);return v=void 0===v?"":v+"",c||notEncode?v:v}catch(e){return console.log("BuildHtml Fail",a+"\n"+e.stack),""}}),loop&&/\{(fn:)?(.+?)\}/.test(html)&&(html=BuildHtml(html,o,notEncode,loop)),html};
function RandomKey(){
Expand Down Expand Up @@ -183,6 +183,8 @@
<div class="pd btns">
<button onclick="recstart()">录制</button>
<button onclick="recstop()">停止</button>

<button onclick="recstopX()" style="margin-left:60px;padding:2px 5px;">停止(仅清理)</button>
</div>
<div class="pd recpower">
<div style="height:40px;width:300px;background:#999;position:relative;">
Expand Down Expand Up @@ -384,6 +386,14 @@
call(RecordApp.Current.Key+"打开失败:"+err);
});
};
function recstopX(){
RecordApp.Stop(
null //success传null就只会清理资源,不会进行转码
,function(msg){
reclog("已清理,错误信息:"+msg);
}
);
};
var recblob={};
function recstop(call){
recstopFn(call,true,function(){
Expand Down
240 changes: 240 additions & 0 deletions assets/runtime-codes/lib.samplerate.raise.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
/******************
《【Demo库】PCM采样率提升》
作者:高坚果
时间:2020-1-9 20:48:39
文档:
Recorder.SampleRaise(pcmDatas,pcmSampleRate,newSampleRate)
pcmDatas: [[Int16,...]] pcm片段列表,二维数组
pcmSampleRate:pcm的采样率
newSampleRate:要转换成的采样率
返回值:{
sampleRate:16000 结果的采样率,>=pcmSampleRate
data:[Int16,...] 转换后的PCM结果
}
本方法将简单的提升pcm的采样率,如果新采样率低于pcm采样率,将不会进行任何处理。采用的简单算法能力有限,会引入能感知到的轻微杂音。
Recorder.SampleData只提供降低采样率,不提供提升采样率,因为由低的采样率转成高的采样率没有存在的意义。提升采样率的代码不会作为核心功能提供,但某些场合确实需要提升采样率,可自行编写代码转换一下即可。
******************/

//=====采样率提升核心函数==========
Recorder.SampleRaise=function(pcmDatas,pcmSampleRate,newSampleRate){
var size=0;
for(var i=0;i<pcmDatas.length;i++){
size+=pcmDatas[i].length;
};

var step=newSampleRate/pcmSampleRate;
if(step<=1){//新采样不高于pcm采样率不处理
step=1;
newSampleRate=pcmSampleRate;
}else{
size=Math.floor(size*step);
};

var res=new Int16Array(size);

//处理数据
var posFloat=0,prev=0;
for (var index=0,nl=pcmDatas.length;index<nl;index++) {
var arr=pcmDatas[index];
for(var i=0;i<arr.length;i++){
var cur=arr[i];

var pos=Math.floor(posFloat);
posFloat+=step;
var end=Math.floor(posFloat);

//简单的从prev平滑填充到cur,有效减少转换引入的杂音
var n=(cur-prev)/(end-pos);
for(var j=1;pos<end;pos++,j++){
//res[pos]=cur;
res[pos]=Math.floor(prev+(j*n));
};

prev=cur;
};
};

return {
sampleRate:newSampleRate
,data:res
};
};



//************测试************
var sampleRaiseInfo=window.sampleRaiseInfo||{from:16000,to:44100};
var transform=function(buffers,sampleRate){
sampleRaiseInfo.buffers=buffers;
sampleRaiseInfo.sampleRate=sampleRate;
if(!buffers){
Runtime.Log("请先录个音",1);
return;
};
var from=sampleRaiseInfo.from;
var to=sampleRaiseInfo.to;

//准备低采样率数据
var pcmFrom=Recorder.SampleData(buffers,sampleRate,from).data;

//转换成高采样率
var pcmTo=Recorder.SampleRaise([pcmFrom],from,to).data;

var mockFrom=Recorder({type:"wav",sampleRate:from}).mock(pcmFrom,from);
mockFrom.stop(function(blob1,duration1){

var mockTo=Recorder({type:"wav",sampleRate:to}).mock(pcmTo,to);
mockTo.stop(function(blob2,duration2){
Runtime.Log(from+"->"+to,2);

Runtime.LogAudio(blob1,duration1,mockFrom,"低采样");
Runtime.LogAudio(blob2,duration2,mockTo,"高采样");
});

});
};
var k8k16=function(){
sampleRaiseInfo.from=8000;
sampleRaiseInfo.to=16000;

transform(sampleRaiseInfo.buffers,sampleRaiseInfo.sampleRate);
};
var k16k441=function(){
sampleRaiseInfo.from=16000;
sampleRaiseInfo.to=44100;

transform(sampleRaiseInfo.buffers,sampleRaiseInfo.sampleRate);
};




//******音频数据源,采集原始音频用的******
//显示控制按钮
Runtime.Ctrls([
{name:"开始录音",click:"recStart"}
,{name:"结束录音",click:"recStop"}
,{html:"<hr/>"}
,{name:"8k转16k",click:"k8k16"}
,{name:"16k转44.1k",click:"k16k441"}
]);


//加载录音框架
Runtime.Import([
{url:RootFolder+"/src/recorder-core.js",check:function(){return !window.Recorder}}
,{url:RootFolder+"/src/engine/wav.js",check:function(){return !Recorder.prototype.wav}}
]);

//调用录音
var rec;
function recStart(){
rec=Recorder({
type:"wav"
,sampleRate:48000
,bitRate:16
,onProcess:function(buffers,powerLevel,bufferDuration,bufferSampleRate){
Runtime.Process.apply(null,arguments);
}
});
var t=setTimeout(function(){
Runtime.Log("无法录音:权限请求被忽略(超时假装手动点击了确认对话框)",1);
},8000);

rec.open(function(){//打开麦克风授权获得相关资源
clearTimeout(t);
rec.start();//开始录音
},function(msg,isUserNotAllow){//用户拒绝未授权或不支持
clearTimeout(t);
Runtime.Log((isUserNotAllow?"UserNotAllow,":"")+"无法录音:"+msg, 1);
});
};
function recStop(){
rec.stop(function(blob,duration){
Runtime.LogAudio(blob,duration,rec);

transform(rec.buffers,rec.srcSampleRate);
},function(msg){
Runtime.Log("录音失败:"+msg, 1);
},true);
};



//*****拖拽或者选择文件******
$(".choiceFileBox").remove();
Runtime.Log('<div class="choiceFileBox">\
<div class="dropFile" onclick="$(\'.choiceFile\').click()" style="border: 3px dashed #a2a1a1;background:#eee; padding:30px 0; text-align:center;cursor: pointer;">\
拖拽多个音乐文件到这里 / 点此选择,并转换\
</div>\
<input type="file" class="choiceFile" style="display:none" accept="audio/*" multiple="multiple">\
</div>');
$(".dropFile").bind("dragover",function(e){
e.preventDefault();
}).bind("drop",function(e){
e.preventDefault();

readChoiceFile(e.originalEvent.dataTransfer.files);
});
$(".choiceFile").bind("change",function(e){
readChoiceFile(e.target.files);
});
function readChoiceFile(files){
if(!files.length){
return;
};

Runtime.Log("发现"+files.length+"个文件,开始转换...");

var idx=-1;
var run=function(){
idx++;
if(idx>=files.length){
return;
};

var file = files[idx];
var reader = new FileReader();
reader.onload = function(e){
decodeAudio(file.name,e.target.result,run);
}
reader.readAsArrayBuffer(file);
};
run();
};
var decodeAudio=function(name,arr,call){
if(!Recorder.Support()){//强制激活Recorder.Ctx 不支持大概率也不支持解码
Runtime.Log("浏览器不支持音频解码",1);
return;
};
var srcBlob=new Blob([arr],{type:"audio/"+(/[^.]+$/.exec(name)||[])[0]});
var ctx=Recorder.Ctx;
ctx.decodeAudioData(arr,function(raw){
var src=raw.getChannelData(0);
var sampleRate=raw.sampleRate;
console.log(name,raw,srcBlob);

var pcm=new Int16Array(src.length);
for(var i=0;i<src.length;i++){//floatTo16BitPCM
var s=Math.max(-1,Math.min(1,src[i]));
s=s<0?s*0x8000:s*0x7FFF;
pcm[i]=s;
};

Runtime.LogAudio(srcBlob,Math.round(src.length/sampleRate*1000),{set:{sampleRate:sampleRate}},"已解码"+name);

rec=null;
transform([pcm],sampleRate);

call();
},function(e){
Runtime.Log("audio解码失败:"+e.message,1);
});
};


Runtime.Log("结束录音转换格式以最后点击的哪个为准");
2 changes: 2 additions & 0 deletions assets/工具-代码运行和静态分发Runtime.html
Original file line number Diff line number Diff line change
Expand Up @@ -908,6 +908,8 @@
,{n:"【教程】实时多路音频混音",k:"teach.realtime.mix_multiple"}
,{n:"【教程】变速变调音频转换",k:"teach.sonic.transform"}

,{n:"【Demo库】PCM采样率提升",k:"lib.samplerate.raise"}

];

var markdown=[];
Expand Down
Loading

0 comments on commit 4cfc685

Please sign in to comment.