search.xml

<?xml version="1.0" encoding="utf-8"?>
<search>
  <entry>
    <title>Effective Modern C++ 行为习惯列举</title>
    <url>/2019/05/07/Effective-Modern-C-%E8%A1%8C%E4%B8%BA%E4%B9%A0%E6%83%AF%E5%88%97%E4%B8%BE/</url>
    <content><![CDATA[<h2 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h2><blockquote>
<p>面向已经掌握了C++11之前的同学, 本文只列举了基本的几条.</p>
<p>ref: Effective Modern C++</p>
</blockquote><h2 id="用auto-代替显示声明"><a href="#用auto-代替显示声明" class="headerlink" title="用auto 代替显示声明."></a>用<strong>auto</strong> 代替显示声明.</h2><p>✘ 错误示范:</p><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typename</span> <span class="built_in">std</span>::iterator_traits&lt;It&gt;::value_type currValue = *b;</span><br></pre></td></tr></table></figure><a id="more"></a>


<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">auto</span> currValue = *b;</span><br></pre></td></tr></table></figure>
<h2 id="使用nullptr-替代NULL-和0"><a href="#使用nullptr-替代NULL-和0" class="headerlink" title="使用nullptr 替代NULL 和0."></a>使用<strong>nullptr</strong> 替代<strong>NULL</strong> 和<strong>0</strong>.</h2><p>✘ 错误示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (result == <span class="number">0</span>)&#123;</span><br><span class="line">  ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (result == <span class="literal">NULL</span>)&#123;</span><br><span class="line">	...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">if</span> (result == <span class="literal">nullptr</span>)&#123;</span><br><span class="line">  ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h2 id="使用using-替代typedef"><a href="#使用using-替代typedef" class="headerlink" title="使用using 替代typedef."></a>使用<strong>using</strong> 替代<strong>typedef</strong>.</h2><p>✘ 错误示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="built_in">std</span>::<span class="built_in">unique_ptr</span>&lt;<span class="built_in">std</span>::<span class="built_in">unordered_map</span>&lt;<span class="built_in">std</span>::<span class="built_in">string</span>, <span class="built_in">std</span>::<span class="built_in">string</span>&gt;&gt; UPtrMapSS;</span><br></pre></td></tr></table></figure>
<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">using</span> UPtrMapSS = <span class="built_in">std</span>::<span class="built_in">unique_ptr</span>&lt;<span class="built_in">std</span>::<span class="built_in">unordered_map</span>&lt;<span class="built_in">std</span>::<span class="built_in">string</span>, <span class="built_in">std</span>::<span class="built_in">string</span>&gt;&gt;;</span><br></pre></td></tr></table></figure>
<h2 id="有范围的enum-替代无范围的enum"><a href="#有范围的enum-替代无范围的enum" class="headerlink" title="有范围的enum 替代无范围的enum."></a>有范围的<strong>enum</strong> 替代无范围的<strong>enum</strong>.</h2><p>✘ 错误示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">enum</span> Color &#123; black, white, red &#125;;</span><br><span class="line"><span class="keyword">auto</span> white = <span class="literal">false</span>; <span class="comment">//white 已经被声明了, error!</span></span><br></pre></td></tr></table></figure>
<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">enum</span> <span class="class"><span class="keyword">class</span> <span class="title">Color</span> &#123;</span> black, white, red &#125;;</span><br><span class="line"><span class="keyword">auto</span> white = <span class="literal">false</span>; <span class="comment">// 一切正常</span></span><br><span class="line">Color c = Color::white; <span class="comment">//规范的声明方式</span></span><br><span class="line"><span class="keyword">auto</span> c = Color::white; <span class="comment">//规范的声明方式</span></span><br></pre></td></tr></table></figure>
<h2 id="禁用函数时-用delete-替代private"><a href="#禁用函数时-用delete-替代private" class="headerlink" title="禁用函数时, 用delete 替代private."></a>禁用函数时, 用<strong>delete</strong> 替代<strong>private</strong>.</h2><p>✘ 错误示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">basic_ios</span> :</span> <span class="keyword">public</span> ios_base &#123; </span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">	...</span><br><span class="line"><span class="keyword">private</span>:</span><br><span class="line">	basic_ios(<span class="keyword">const</span> basic_ios&amp; ); </span><br><span class="line">	basic_ios&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> basic_ios&amp;); </span><br><span class="line">  ...</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure>
<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">basic_ios</span> :</span> <span class="keyword">public</span> ios_base &#123; </span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">	basic_ios(<span class="keyword">const</span> basic_ios&amp; ) = <span class="keyword">delete</span>; </span><br><span class="line">	basic_ios&amp; <span class="keyword">operator</span>=(<span class="keyword">const</span> basic_ios&amp;) = <span class="keyword">delete</span>; </span><br><span class="line">  ...</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure>
<h2 id="使用override-关键字标注-override函数"><a href="#使用override-关键字标注-override函数" class="headerlink" title="使用override 关键字标注 override函数."></a>使用<strong>override</strong> 关键字标注 override函数.</h2><figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Base</span> &#123;</span> </span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf1</span><span class="params">()</span> <span class="keyword">const</span></span>; </span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf2</span><span class="params">(<span class="keyword">int</span> x)</span></span>; </span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf3</span><span class="params">()</span> &amp;</span>; </span><br><span class="line">  <span class="function"><span class="keyword">void</span> <span class="title">mf4</span><span class="params">()</span> <span class="keyword">const</span></span>; </span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Derived</span>:</span> <span class="keyword">public</span> Base &#123; </span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf1</span><span class="params">()</span> override</span>; </span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf2</span><span class="params">(<span class="keyword">unsigned</span> <span class="keyword">int</span> x)</span> override</span>; </span><br><span class="line">  <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf3</span><span class="params">()</span> &amp;&amp; override</span>; <span class="function"><span class="keyword">virtual</span> <span class="keyword">void</span> <span class="title">mf4</span><span class="params">()</span> <span class="keyword">const</span> override</span>;</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure>
<h2 id="使用const-iterators-替代iterators"><a href="#使用const-iterators-替代iterators" class="headerlink" title="使用const_iterators 替代iterators."></a>使用const_iterators 替代iterators.</h2><p>✘ 错误示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="built_in">std</span>::<span class="built_in">vector</span>&lt;<span class="keyword">int</span>&gt; values;</span><br><span class="line">…</span><br><span class="line"><span class="keyword">auto</span> it = <span class="built_in">std</span>::find(values.begin(),values.end(), <span class="number">1983</span>); <span class="comment">//使用begin()和end()</span></span><br><span class="line">values.insert(it, <span class="number">1998</span>);</span><br></pre></td></tr></table></figure>
<p>✔ 正确示范:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="built_in">std</span>::<span class="built_in">vector</span>&lt;<span class="keyword">int</span>&gt; values; </span><br><span class="line">…</span><br><span class="line"><span class="keyword">auto</span> it = <span class="built_in">std</span>::find(values.cbegin(),values.cend(), <span class="number">1983</span>);<span class="comment">//使用cbegin()和cend()</span></span><br><span class="line">values.insert(it, <span class="number">1998</span>);</span><br></pre></td></tr></table></figure>
<h2 id="如果函数不会抛出异常-使用noexcept进行声明"><a href="#如果函数不会抛出异常-使用noexcept进行声明" class="headerlink" title="如果函数不会抛出异常, 使用noexcept进行声明."></a>如果函数不会抛出异常, 使用<strong>noexcept</strong>进行声明.</h2><p>✘ C++98:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">f</span><span class="params">(<span class="keyword">int</span> x)</span> <span class="title">throw</span><span class="params">()</span></span>;</span><br></pre></td></tr></table></figure>
<p>✔ C++11:</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">f</span><span class="params">(<span class="keyword">int</span> x)</span> <span class="keyword">noexcept</span></span>;</span><br></pre></td></tr></table></figure>
<h2 id="使用智能指针-std-unique-ptr-std-shared-ptr-std-weak-ptr替代传统指针-std-auto-淘汰了别用了"><a href="#使用智能指针-std-unique-ptr-std-shared-ptr-std-weak-ptr替代传统指针-std-auto-淘汰了别用了" class="headerlink" title="使用智能指针 std::unique_ptr, std::shared_ptr,  std::weak_ptr替代传统指针 (std::auto 淘汰了别用了)."></a>使用智能指针 <code>std::unique_ptr</code>, <code>std::shared_ptr</code>,  <code>std::weak_ptr</code>替代传统指针 (<code>std::auto</code> 淘汰了别用了).</h2><h2 id="能用constexpr就用constexpr"><a href="#能用constexpr就用constexpr" class="headerlink" title="能用constexpr就用constexpr."></a>能用constexpr就用constexpr.</h2><h2 id="让常成员函数线程安全-使用std-mutex-或std-atomic-等"><a href="#让常成员函数线程安全-使用std-mutex-或std-atomic-等" class="headerlink" title="让常成员函数线程安全: 使用std::mutex 或std::atomic 等."></a>让<em>常成员函数</em>线程安全: 使用<code>std::mutex</code> 或<code>std::atomic</code> 等.</h2><h2 id="善用右值-Rvalue-语义转移-Move-Semantics-完美转发-Perfect-Forwarding"><a href="#善用右值-Rvalue-语义转移-Move-Semantics-完美转发-Perfect-Forwarding" class="headerlink" title="善用右值[Rvalue], 语义转移[Move Semantics], 完美转发[Perfect Forwarding]"></a>善用右值[<em>Rvalue</em>], 语义转移[<em>Move Semantics</em>], 完美转发[<em>Perfect Forwarding</em>]</h2><h2 id="善用Lambda-表达式"><a href="#善用Lambda-表达式" class="headerlink" title="善用Lambda 表达式"></a>善用Lambda 表达式</h2><h2 id="善用并发编程API"><a href="#善用并发编程API" class="headerlink" title="善用并发编程API"></a>善用并发编程API</h2><h2 id="容器中使用emplace-back-替代push-back"><a href="#容器中使用emplace-back-替代push-back" class="headerlink" title="容器中使用emplace_back(),  替代push_back()"></a>容器中使用<code>emplace_back()</code>,  替代<code>push_back()</code></h2>]]></content>
      <categories>
        <category>c++</category>
      </categories>
      <tags>
        <tag>c++</tag>
      </tags>
  </entry>
  <entry>
    <title>Hello World</title>
    <url>/2018/12/23/hello-world/</url>
    <content><![CDATA[<p>Hello World!</p>
<p>我回来啦~❕❕❕❕<br>开始写作,更新blog啦 🎉🎉🎉🎉</p>
]]></content>
      <categories>
        <category>life</category>
      </categories>
      <tags>
        <tag>life</tag>
      </tags>
  </entry>
  <entry>
    <title>MySQL Note</title>
    <url>/2020/07/31/MySQL-Note/</url>
    <content><![CDATA[<h1 id="高性能MySQL-笔记"><a href="#高性能MySQL-笔记" class="headerlink" title="高性能MySQL 笔记"></a>高性能MySQL 笔记</h1><h2 id="表"><a href="#表" class="headerlink" title="表"></a>表</h2><h3 id="列"><a href="#列" class="headerlink" title="列"></a>列</h3><h4 id="过多的列"><a href="#过多的列" class="headerlink" title="过多的列"></a>过多的列</h4><p>由于MySQL的存储层与通信层之间使用的缓冲格式是列缓冲的, 过多的列转行导致开销增大.</p><h3 id="范式"><a href="#范式" class="headerlink" title="范式"></a>范式</h3><ul>
<li>1NF: 列具有原子性,  不可拆分.</li>
<li>2NF: 在满足第一范式（1NF）的基础上, 每一个非码属性（不在主键中的列）都必须<strong>完全函数依赖于</strong> <em>候选码</em>. 若某些非主属性不依赖于候选码, 那么会不符合2NF.</li>
<li>3NF: 在满足第二范式（2NF）的基础上, 每个非主属性不依赖于其它非主属性, 即在2NF基础上，消除非码属性对候选码的传递函数依赖.(即所有非主属性对主属性直接依赖, 不能传递地依赖.)</li>
<li>BCNF: 非主属性不能对主属性部分依赖.</li>
</ul><a id="more"></a>

<h3 id="数据类型"><a href="#数据类型" class="headerlink" title="数据类型"></a>数据类型</h3><h4 id="VARCHAR-vs-CHAR"><a href="#VARCHAR-vs-CHAR" class="headerlink" title="VARCHAR vs. CHAR"></a>VARCHAR vs. CHAR</h4><ul>
<li>varchar 保留尾部空格, char不保留.</li>
<li>varchar 适用于字符串长度方差大的.</li>
<li>varchar 省空间但是update成本高.</li>
</ul>
<h4 id="BLOB-amp-TEXT"><a href="#BLOB-amp-TEXT" class="headerlink" title="BLOB &amp; TEXT"></a>BLOB &amp; TEXT</h4><ul>
<li>都是存储指针, 需要二次访问.</li>
<li>可以做索引, 但是必须给出前缀长度.</li>
</ul>
<h2 id="事务"><a href="#事务" class="headerlink" title="事务"></a>事务</h2><h3 id="A-C-I-D"><a href="#A-C-I-D" class="headerlink" title="A.C.I.D."></a>A.C.I.D.</h3><ul>
<li>Atomicity（原子性）：一个事务（transaction）中的所有操作，或者全部完成，或者全部不完成，不会结束在中间某个环节。事务在执行过程中发生错误，会被回滚（Rollback）到事务开始前的状态，就像这个事务从来没有执行过一样。即，事务不可分割、不可约简。</li>
<li>Consistency（一致性）：在事务开始之前和事务结束以后，数据库的完整性没有被破坏。这表示写入的资料必须完全符合所有的预设约束、触发器、级联回滚等。</li>
<li>Isolation（隔离性）：数据库允许多个并发事务同时对其数据进行读写和修改的能力，隔离性可以防止多个事务并发执行时由于交叉执行而导致数据的不一致。事务隔离分为不同级别，包括未提交读（Read uncommitted）、提交读（read committed）、可重复读（repeatable read）和串行化（Serializable）。</li>
<li>Durability（持久性）：事务处理结束后，对数据的修改就是永久的，即便系统故障也不会丢失。</li>
</ul>
<table>
<thead>
<tr>
<th>出现的问题</th>
<th>原因</th>
<th>解决</th>
</tr>
</thead>
<tbody>
<tr>
<td>脏读</td>
<td>读取了其他事务未提交的数据, 导致数据错误.</td>
<td>改为提交读. —— 只能读取事务提交后的数据.</td>
</tr>
<tr>
<td>不可重复读</td>
<td>一个对同一数据的两次查询之间, 另一个事务做出了修改, 导致前后不一致.</td>
<td>改为可重复读. —— 使用 X, S锁.</td>
</tr>
<tr>
<td>幻读</td>
<td>查询结果为多个记录的时候, 前后两次这样的查询之间, 插入了新的记录, 导致前后结果不一致</td>
<td>串行化 或 Next-Key Lock</td>
</tr>
<tr>
<td>更新丢失</td>
<td>两个事务同时对同一个数据进行更新操作, 其中在read-and-write阻塞, 尽管加S, X锁仍然导致错误.</td>
<td>串行化.</td>
</tr>
</tbody>
</table>
<blockquote>
<p>更新丢失出现, 有可能并非事务隔离能避免的.</p>
<p>例如现在两个程序端, 同时对同一个人取款. 初始1000, 事务一取 100块, 事务二取 1 块.</p>
<p>共计取了101, 应该为899元, 结果却为999元.</p>
<p>串行化可以解决此类问题.</p>
</blockquote>
<p><img src="/images/Mysql_Note/0.png" alt="事务"></p>
<h4 id="事务分类"><a href="#事务分类" class="headerlink" title="事务分类"></a>事务分类</h4><ul>
<li><p>扁平事务</p>
</li>
<li><p>带保存点的扁平事务</p>
</li>
<li><p>链事务</p>
</li>
<li><p>嵌套事务</p>
</li>
</ul>
<h4 id="事务陷阱"><a href="#事务陷阱" class="headerlink" title="事务陷阱"></a>事务陷阱</h4><ul>
<li>自动回滚 —— 自动回滚不会抛出异常, 无法知道异常. (SQL Server 会抛出异常, 这个是MySQL特有的缺点)</li>
<li>自动提交.</li>
<li>在循环中提交 —— 意外导致在未知位置终止, 无法还原.</li>
</ul>
<h3 id="锁"><a href="#锁" class="headerlink" title="锁"></a>锁</h3><h4 id="行锁"><a href="#行锁" class="headerlink" title="行锁"></a>行锁</h4><h5 id="共享锁-独占锁"><a href="#共享锁-独占锁" class="headerlink" title="共享锁, 独占锁"></a>共享锁, 独占锁</h5><ul>
<li>允许多个事务读同一行.</li>
<li>当事务对同一行进行写操作, 将阻塞其他所有事务的读、写操作.</li>
</ul>
<h4 id="行锁算法"><a href="#行锁算法" class="headerlink" title="行锁算法"></a>行锁算法</h4><h5 id="Record-Lock"><a href="#Record-Lock" class="headerlink" title="Record Lock"></a>Record Lock</h5><p>单一行锁.</p>
<h5 id="Gap-Lock"><a href="#Gap-Lock" class="headerlink" title="Gap Lock"></a>Gap Lock</h5><p>锁定一个区间, 但是不包括该记录本身.</p>
<h5 id="Next-Key-Lock"><a href="#Next-Key-Lock" class="headerlink" title="Next-Key Lock"></a>Next-Key Lock</h5><p>锁定该记录本身, 并且锁定包含它的一个范围.</p>
<h4 id="粒度锁"><a href="#粒度锁" class="headerlink" title="粒度锁"></a>粒度锁</h4><h5 id="意向锁-树状锁"><a href="#意向锁-树状锁" class="headerlink" title="意向锁 :  树状锁"></a>意向锁 :  树状锁</h5><p>锁的结构呈树状, 粗粒度锁拥有几个粒度细的锁.</p>
<p>叶子的粒度细, 父节点的粒度粗.</p>
<h2 id="索引"><a href="#索引" class="headerlink" title="索引"></a>索引</h2><h3 id="索引的相关理论"><a href="#索引的相关理论" class="headerlink" title="索引的相关理论"></a>索引的相关理论</h3><h4 id="最左前缀匹配"><a href="#最左前缀匹配" class="headerlink" title="最左前缀匹配"></a>最左前缀匹配</h4><p>MySQL引擎采用最左前缀匹配的机制, 可以高效从左匹配.</p>
<p>所以最左端是模糊查询的时候, 最左前缀匹配失效, 导致性能降低.</p>
<h4 id="可选择性"><a href="#可选择性" class="headerlink" title="可选择性"></a>可选择性</h4><p>可选择性的意思是, 针对某一字段查询出的结果, 对于这个的分区度.(这一字段的重复程度)</p>
<blockquote>
<p>例如, 性别的区分度就很低: 根据某个性别查询出的结果, 性别的区分度及其低. </p>
<p>再如, 年龄、城市的区分度稍稍高于性别的区分度.</p>
<p>最高的区分度莫过于ID之类的字段. 其是唯一的, 所以它的可选择性及高.</p>
</blockquote>
<p>解决可选择性低的方法是, 将几个可选择性低的字段使用多列索引组合起来, 让其可区分度上升.</p>
<h4 id="聚集索引-vs-辅助索引"><a href="#聚集索引-vs-辅助索引" class="headerlink" title="聚集索引 vs. 辅助索引"></a>聚集索引 vs. 辅助索引</h4><ul>
<li>聚集索引: 也称主码索引, 使用主码建立的索引, 直接指向数据.</li>
<li>辅助索引: 非主码索引, 该索引指向主码. 先用该索引定位主码, 再用主码定位数据.</li>
</ul>
<h3 id="聚簇索引"><a href="#聚簇索引" class="headerlink" title="聚簇索引"></a>聚簇索引</h3><p>将某一列相同相近的记录排在一起, 进行索引.</p>
<blockquote>
<p>其提高了I/O密集型的性能, 相关数据保存在一起.</p>
</blockquote>
<h4 id="聚簇-vs-可选择性-——-UUID-vs-自增ID的选择及UUID的优化"><a href="#聚簇-vs-可选择性-——-UUID-vs-自增ID的选择及UUID的优化" class="headerlink" title="聚簇 vs 可选择性 —— UUID vs. 自增ID的选择及UUID的优化"></a>聚簇 vs 可选择性 —— UUID vs. 自增ID的选择及UUID的优化</h4><ul>
<li>UUID应该使用UNHEX存储在Binary(16)中, 而不是用字符串存储.</li>
<li>UUID的引入, 导致了索引的随机性大大增加, 索引占用空间变大.</li>
<li>因为UUID是随机的,  每一页都是随机填充的, 导致页内碎片变多. 而且因为写入是乱序的, 导致频繁的页分裂.</li>
<li>UUID是随机的, 导致大量的随机I/O, 性能大幅下降.</li>
<li>自增ID会有自增锁, 频繁的锁也会导致阻塞. </li>
<li>自增PK会成为热点数据, 并发插入导致<strong>间隙竞争</strong>可以考虑修改<code>innodb_autoinc_lock_mode</code> 进行优化.</li>
</ul>
<blockquote>
<p> 折中方案:</p>
<p>使用前缀顺序+后缀随机的组合式ID可以达到这两种极端的综合.</p>
</blockquote>
<h3 id="多列索引"><a href="#多列索引" class="headerlink" title="多列索引"></a>多列索引</h3><p>为了解决可选择性过低的问题, 多个选择性低的字段, 可用多列索引组合起来, 提高可选择性, 这就是多列索引.</p>
<p>但是选择性高的字段组合起来, 会增加负担.</p>
<blockquote>
<p>例如有一些待查询的列: &lt;性别,  年龄, 城市&gt;</p>
<ul>
<li>性别的可选择性最低, 因为查询出的人性别无非就几种, 这列的查询结果重复度极高.</li>
<li>其次, 年龄一般在0~99之间集中分布,  重复读虽然低于性别, 但是几十万的数据集中分布在0~99这个区间, 重复度也不低.</li>
<li>同理, 城市的区分度高于前两者, 但是可选择性也是一般.</li>
</ul>
<p>此时, 可以建立一个多列索引, 将这三个可选择性低的列绑成一个多列索引, 可以大大提高可选择性.</p>
</blockquote>
<h3 id="覆盖索引"><a href="#覆盖索引" class="headerlink" title="覆盖索引***"></a>覆盖索引***</h3><blockquote>
<p>如果一个索引覆盖了查询字段的所有值, 那么称其为覆盖索引.</p>
</blockquote>
<p>优点:</p>
<ul>
<li>索引条目少.</li>
<li>索引按照列值存储, 使用顺序的磁盘I/O; 少次的顺序I/O比多次的随机I/O性能好.</li>
</ul>
<p><strong>索引失效情况</strong></p>
<ul>
<li>索引没有覆盖到查询列. —— <strong>使用延迟查询解决.</strong></li>
<li>MySQL5.5 之后可以使用最左前缀匹配的Like查询, 但是非最左前缀匹配的模糊查询将使得索引失效.</li>
<li>ORDER BY, GROUP BY所用的列未索引覆盖, 导致无法索引排序.</li>
</ul>
<h3 id="索引排序"><a href="#索引排序" class="headerlink" title="索引排序***"></a>索引排序***</h3><blockquote>
<p>利用索引对数据进行排序.</p>
</blockquote>
<p>针对 ORDER BY和GROUP BY</p>
<ul>
<li>只有当查询的列顺序和排序的列顺序相同, 且排序方向相同(都是升序或都是降序), MySQL才会对其进行索引排序.</li>
<li>关联多表的ORDER BY、GROUP BY引用的字段必须全部为首张表, 才会触发索引排序.</li>
<li>必须满足最左前缀匹配.</li>
<li>范围查询中的列与ORDER BY、GROUP BY中的列要一致, 才能出发索引排序.</li>
</ul>
<h3 id="冗余索引"><a href="#冗余索引" class="headerlink" title="冗余索引"></a>冗余索引</h3><blockquote>
<p>若有索引$(A,B)$ 之后又出现了 索引$(A)$ , 则 $(A)$ 是$(A,B)$的索引.</p>
<p>但$(A)$不是$(B,A)$ 的索引.</p>
</blockquote>
<p>因为索引多, 虽然查询快, 但是插入慢(需要维护索引的成本变高). 冗余、重复的索引会降低性能.</p>
<h3 id="索引类型"><a href="#索引类型" class="headerlink" title="索引类型"></a>索引类型</h3><blockquote>
<p>EXPLAIN中的extra字段可显示. 指检索数据的方式.</p>
</blockquote>
<table>
<thead>
<tr>
<th>Extra</th>
<th>意思</th>
<th>可能出现的情况</th>
</tr>
</thead>
<tbody>
<tr>
<td>using index</td>
<td>使用索引</td>
<td>覆盖索引, 高效.</td>
</tr>
<tr>
<td>using where</td>
<td>存储引擎收到记录后进行过滤</td>
<td>索引未覆盖, 或部分覆盖.</td>
</tr>
<tr>
<td>using condition</td>
<td>使用条件</td>
<td>查询会先条件过滤索引，过滤完索引后找到所有符合索引条件的数据行，<br>随后用 WHERE 子句中的其他条件去过滤这些数据行.</td>
</tr>
<tr>
<td>using filesort</td>
<td>使用文件排序</td>
<td>无法利用索引完成的排序操作称为“文件排序”.</td>
</tr>
<tr>
<td>using temporary</td>
<td>使用临时表</td>
<td>需要使用临时表来存储结果集，常见于排序和分组查询.</td>
</tr>
</tbody>
</table>
<h3 id="三星索引"><a href="#三星索引" class="headerlink" title="三星索引"></a>三星索引</h3><table>
<thead>
<tr>
<th>星级</th>
<th>定义</th>
<th>原理</th>
<th>实现</th>
</tr>
</thead>
<tbody>
<tr>
<td>🌟</td>
<td>这个索引与一个查询相关的索引行是相邻的, 或者是相近的</td>
<td>它最小化了必须扫描的索引片的宽度</td>
<td>把 WHERE 后的等值条件列作为索引最开头的列</td>
</tr>
<tr>
<td>🌟🌟</td>
<td>索引行的顺序与查询语句的需求一致</td>
<td>排除了排序操作</td>
<td>ORDER BY的列都在索引中, 且与查询顺序一致</td>
</tr>
<tr>
<td>🌟🌟🌟</td>
<td>如果索引行中包含查询语句中的所有列</td>
<td>覆盖索引</td>
<td>实现覆盖索引</td>
</tr>
</tbody>
</table>
<h2 id="查询"><a href="#查询" class="headerlink" title="查询"></a>查询</h2><blockquote>
<p>查询变慢的原因包括但不限于:</p>
<ul>
<li>返回了过多的数据.</li>
<li>返回了不需要的列.</li>
</ul>
</blockquote>
<h3 id="扫描类型"><a href="#扫描类型" class="headerlink" title="扫描类型"></a>扫描类型</h3><blockquote>
<p>使用EXPLAIN语句可以查看该查询的扫描类型, 在type字段. 指检索数据的集合.</p>
<p>性能高低:</p>
<p>常数引用  &gt; 唯一索引查询 &gt; 范围扫描 &gt; 索引扫描 &gt; 全表扫描</p>
</blockquote>
<p>扫描类型</p>
<table>
<thead>
<tr>
<th style="text-align:center">类型</th>
<th style="text-align:center">名称</th>
<th>成因</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center">ALL</td>
<td style="text-align:center">全表扫描</td>
<td>查询扫描了全部整个表, 性能最低</td>
</tr>
<tr>
<td style="text-align:center">index</td>
<td style="text-align:center">索引全扫描</td>
<td>查询扫描了整个索引</td>
</tr>
<tr>
<td style="text-align:center">range</td>
<td style="text-align:center">索引范围扫描</td>
<td>是范围查询</td>
</tr>
<tr>
<td style="text-align:center">ref</td>
<td style="text-align:center">非唯一索引扫描</td>
<td>非唯一索引或者使用了最左前缀匹配</td>
</tr>
<tr>
<td style="text-align:center">eq_ref</td>
<td style="text-align:center">唯一索引扫描</td>
<td>一般出现在多表连接时使用primary key或者unique index作为关联条件</td>
</tr>
<tr>
<td style="text-align:center">const,system</td>
<td style="text-align:center">单表最多有一个匹配行</td>
<td>出现在根据主键primary key, 或者唯一索引 unique index 进行的查询</td>
</tr>
<tr>
<td style="text-align:center">NULL</td>
<td style="text-align:center">不用扫描表或索引</td>
<td>不用访问表或者索引，直接就能够得到结果</td>
</tr>
</tbody>
</table>
<h3 id="简单查询-vs-复杂查询"><a href="#简单查询-vs-复杂查询" class="headerlink" title="简单查询 vs. 复杂查询"></a>简单查询 vs. 复杂查询</h3><p>如果条件允许, 应该使用多个简单查询替代一个复杂查询. </p>
<p>这样的查询结果很高效, 而且还能将查询结果缓存.</p>
<h3 id="关联查询"><a href="#关联查询" class="headerlink" title="关联查询"></a>关联查询</h3><p>MySQL的关联查询是一个左长臂二叉树, 而不是AVL树. 大量的联合查询导致左子树呈现出线性的特征, 导致效率低下.</p>
<p>需要避免过多的多表查询, 或者手动优化JOIN的方式, 使查询树的高度变低.</p>
<p>​                   JOIN</p>
<p>​                /          \</p>
<p>​            JOIN     table4</p>
<p>​          /        \</p>
<p>​     JOIN     table3</p>
<p>   /        \</p>
<p>table1  table2</p>
<h3 id="延迟关联"><a href="#延迟关联" class="headerlink" title="延迟关联"></a>延迟关联</h3><p>若在查询中有索引无法覆盖的列, 应该拆分出来成为一个子查询, 使用例如left outer/inner join进行连接后查询.</p>
<blockquote>
<p>例如:</p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">&gt; SELECT id, description from tableA order by title limit 50,5; -- title没有索引覆盖</span><br><span class="line">&gt; -- 化为</span><br><span class="line">&gt; SELECT id, decription from tableA inner join (SELECT id from tableA order by title) as tmp USING(id); -- 缩小非索引覆盖的范围. 子查询只查pk.</span><br><span class="line">&gt; </span><br><span class="line">&gt; SELECT id, cu_id, name, info, biz_type, gmt_create, gmt_modified,start_time, end_time, market_type, back_leaf_category,item_status,picuture_url FROM relation where biz_type =&apos;0&apos; AND end_time &gt;=&apos;2014-05-29&apos; ORDER BY id asc LIMIT 149420 ,20;</span><br><span class="line">&gt; </span><br><span class="line">&gt; SELECT a.* FROM relation a, (select id from relation where biz_type =&apos;0&apos; AND end_time &gt;=&apos;2014-05-29&apos; ORDER BY id asc LIMIT 149420 ,20 ) b where using(id)</span><br><span class="line">&gt;</span><br></pre></td></tr></table></figure>
</blockquote>
<blockquote>
</blockquote>
<h3 id="可选择性-amp-WHERE的优化"><a href="#可选择性-amp-WHERE的优化" class="headerlink" title="可选择性 &amp; WHERE的优化"></a>可选择性 &amp; WHERE的优化</h3><ul>
<li><p>可选择性高的列放在前查询, 查询结果基数降低, 可以提高性能.</p>
</li>
<li><p>所以, WHERE <strong>精确值查询</strong>{可选择性高, …, 可选择性低}  + <strong>范围查询</strong>{可选择性高(例如IN语句), &gt;&lt; !=,…, 可选择性低(区分度低的列)}</p>
</li>
<li><p>索引未覆盖的列应该进行拆分, 进行延迟关联.</p>
</li>
</ul>
<h3 id="范围查询"><a href="#范围查询" class="headerlink" title="范围查询"></a>范围查询</h3><p>避免多个范围查询: MySQL无法同时使用两个范围查询的索引.</p>
<p>使用IN()替代范围查询, 会提高效率.</p>
<h3 id="IN-amp-EXIST"><a href="#IN-amp-EXIST" class="headerlink" title="IN() &amp; EXIST()"></a>IN() &amp; EXIST()</h3><ul>
<li>EXITS的可以使得查询提前结束, 从而提高效率.</li>
<li>使用IN效率高于比较符号. 但是IN的代价很高, IN()中的值最好不要过多. 因为IN()会对其内部的值进行一个全排列. 例如IN()中写了4个值, 那么其就要对这4个值做一个全排列$4! = 24$ , 可以看出, 这个代价是连乘积复杂度, IN()也并非银弹.</li>
<li>OUTER JOIN的效率 &gt; EXITS的效率 &gt; INNER JOIN的效率.</li>
</ul>
<h3 id="子查询"><a href="#子查询" class="headerlink" title="子查询"></a>子查询</h3><ul>
<li>由于MySQL引擎的原因, 子查询效果很差, 使用left outer join的方式替代, 可以提升性能.</li>
<li>IN()中的子查询效率比EXIST()的子查询低很多, 最好别用IN子查询, 应该优先使用EXIST子查询. (MySQL查询引擎缺陷导致的问题)</li>
</ul>
<h3 id="UNION优化"><a href="#UNION优化" class="headerlink" title="UNION优化"></a>UNION优化</h3><ul>
<li>order by / limit语句写在每个union的子查询中, 而不是最后再order by / limit.</li>
</ul>
<h3 id="LIMIT优化"><a href="#LIMIT优化" class="headerlink" title="LIMIT优化"></a>LIMIT优化</h3><ul>
<li>延迟关联</li>
<li><del>使用between and 语句代替比较符号.</del></li>
</ul>
<h2 id="分区"><a href="#分区" class="headerlink" title="分区"></a>分区</h2><blockquote>
<p>分区也并非银弹.</p>
</blockquote>
<ul>
<li>所有对分区的操作, 第一步都会锁住分区的底层表, 然后判定操作范围, 解锁操作范围之外的分区, 再做相应的操作,.</li>
<li>B-Tree索引将失效.</li>
<li>索引维护极高.</li>
</ul>
]]></content>
  </entry>
  <entry>
    <title>PM2.5 预测</title>
    <url>/2018/12/24/PM2-5/</url>
    <content><![CDATA[<h1 id="使用Linear-Regression-对PM2-5进行预测"><a href="#使用Linear-Regression-对PM2-5进行预测" class="headerlink" title="使用Linear Regression 对PM2.5进行预测"></a>使用Linear Regression 对PM2.5进行预测</h1><h2 id="数据集"><a href="#数据集" class="headerlink" title="数据集"></a>数据集</h2><blockquote>
<p><a href="https://github.com/VIXNESS/machine-learning-course/blob/master/pm25_predict/train.csv" target="_blank" rel="noopener">training data</a></p>
<p><a href="https://github.com/VIXNESS/machine-learning-course/blob/master/pm25_predict/test.csv" target="_blank" rel="noopener">testing data: samples</a></p>
<p><a href="https://github.com/VIXNESS/machine-learning-course/blob/master/pm25_predict/ans.csv" target="_blank" rel="noopener">testing data: label</a></p>
<p>Training data 和 Public testing data 的组织形式:</p>
<p>一天由18行组成,一行为一个指标,一共由18个指标,从第4列开始记录每个指标一天内24小时的变化数值,每个月连续记录前20天作为training set,后10天作为testing set,一共记录了240个小时</p>
</blockquote><a id="more"></a>
<table>
<thead>
<tr>
<th>日期</th>
<th>观测站</th>
<th>指标</th>
<th>0时</th>
<th>…</th>
<th>23时</th>
</tr>
</thead>
<tbody>
<tr>
<td>day 1</td>
<td>xxx</td>
<td>PM2.5</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 1</td>
<td>xxx</td>
<td>PM10</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 1</td>
<td>xxx</td>
<td>SO2</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 1</td>
<td>xxx</td>
<td>…</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 2</td>
<td>xxx</td>
<td>PM2.5</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 2</td>
<td>xxx</td>
<td>PM10</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 2</td>
<td>xxx</td>
<td>SO2</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>day 2</td>
<td>xxx</td>
<td>…</td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
<h2 id="数据处理"><a href="#数据处理" class="headerlink" title="数据处理"></a>数据处理</h2><h3 id="必要类库"><a href="#必要类库" class="headerlink" title="必要类库"></a>必要类库</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> tensorflow <span class="keyword">as</span> tf</span><br><span class="line"><span class="keyword">from</span> tensorflow <span class="keyword">import</span> keras</span><br><span class="line"><span class="keyword">import</span> csv</span><br><span class="line"><span class="keyword">import</span> sys</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br></pre></td></tr></table></figure>
<h3 id="载入数据"><a href="#载入数据" class="headerlink" title="载入数据"></a>载入数据</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">data = []</span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">18</span>):</span><br><span class="line">    data.append([]) <span class="comment"># 初始化18列</span></span><br><span class="line">n_row = <span class="number">0</span></span><br><span class="line"><span class="keyword">with</span> open(<span class="string">'train.csv'</span>,<span class="string">'r'</span>,encoding = <span class="string">'big5'</span>) <span class="keyword">as</span> text: <span class="comment">#csv编码是big5</span></span><br><span class="line">    row = csv.reader(text, delimiter = <span class="string">","</span>)</span><br><span class="line">    <span class="keyword">for</span> r <span class="keyword">in</span> row:</span><br><span class="line">        <span class="keyword">if</span> n_row != <span class="number">0</span>:</span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">3</span>,<span class="number">27</span>):</span><br><span class="line">                <span class="keyword">if</span> r[i] != <span class="string">"NR"</span>: <span class="comment">#NR为未降雨,对其设置为降雨量</span></span><br><span class="line">                    data[(n_row - <span class="number">1</span>) % <span class="number">18</span>].append(float(r[i]))</span><br><span class="line">                <span class="keyword">else</span>:</span><br><span class="line">                	<span class="comment">#设施0降雨为一个接近0的小量,</span></span><br><span class="line">                    <span class="comment">#若设为0,后续的梯度计算会有除0的风险</span></span><br><span class="line">                    data[(n_row - <span class="number">1</span>) % <span class="number">18</span>].append(float(<span class="number">0.0001</span>))</span><br><span class="line">        n_row += <span class="number">1</span></span><br></pre></td></tr></table></figure>
<h3 id="重新组织数据"><a href="#重新组织数据" class="headerlink" title="重新组织数据"></a>重新组织数据</h3><blockquote>
<p>将之前的数据重新组织,对每个小时进行连续拼接</p>
</blockquote>
<table>
<thead>
<tr>
<th>Features</th>
<th>0时</th>
<th>…</th>
<th>23时</th>
<th>0时(次日)</th>
<th>…</th>
<th>23时</th>
</tr>
</thead>
<tbody>
<tr>
<td>PM2.5</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>…</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>..</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>PM 10</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">x = [] <span class="comment"># 样本矩阵</span></span><br><span class="line">y = [] <span class="comment"># 实际的值</span></span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">12</span>): <span class="comment"># 12个月</span></span><br><span class="line">    <span class="keyword">for</span> j <span class="keyword">in</span> range(<span class="number">471</span>): </span><br><span class="line">    <span class="comment"># 每输入9个小时的数值,预测第10个小时的PM2.5值,</span></span><br><span class="line">    <span class="comment"># 这样连续的「10个小时」每个月有471个</span></span><br><span class="line">        x.append([]) <span class="comment">#</span></span><br><span class="line">        <span class="keyword">for</span> w <span class="keyword">in</span> range(<span class="number">18</span>): <span class="comment"># 遍历18个特征</span></span><br><span class="line">            <span class="keyword">for</span> t <span class="keyword">in</span> range(<span class="number">9</span>): <span class="comment"># 遍历前9个小时</span></span><br><span class="line">                x[<span class="number">471</span> * i + j].append(data[w][<span class="number">480</span> * i + j + t])</span><br><span class="line">        <span class="comment"># 将第10个小时的值作为实际的PM2.5的值</span></span><br><span class="line">        y.append(data[<span class="number">9</span>][<span class="number">480</span> * i + j + <span class="number">9</span>])</span><br><span class="line">x = np.array(x)</span><br><span class="line">y = np.array(y)</span><br><span class="line"></span><br><span class="line"><span class="comment">#在第一列添上一条全为1的列作为bias</span></span><br><span class="line">x = np.concatenate((np.ones((x.shape[<span class="number">0</span>],<span class="number">1</span>)),x),axis = <span class="number">1</span>) </span><br><span class="line">w = np.zeros(x.shape[<span class="number">1</span>]) <span class="comment">#weight</span></span><br></pre></td></tr></table></figure>
<h2 id="训练"><a href="#训练" class="headerlink" title="训练"></a>训练</h2><h3 id="定义loss-function"><a href="#定义loss-function" class="headerlink" title="定义loss function"></a>定义loss function</h3><blockquote>
<p>使用 error square<br><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">lossFunction</span><span class="params">(target,weight,samples)</span>:</span></span><br><span class="line">    M = target - np.dot(weight,samples.T)</span><br><span class="line">    loss = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> m <span class="keyword">in</span> M:</span><br><span class="line">        loss += m**<span class="number">2</span></span><br><span class="line">    <span class="keyword">return</span> loss</span><br></pre></td></tr></table></figure></p>
</blockquote>
<h3 id="Gradient-Descent"><a href="#Gradient-Descent" class="headerlink" title="Gradient Descent"></a>Gradient Descent</h3><blockquote>
<p>使用Adagra 对learning rate进行控制</p>
</blockquote>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">lr = <span class="number">8</span> <span class="comment">#learning rate 设置</span></span><br><span class="line">pre_grad = np.ones(x.shape[<span class="number">1</span>])<span class="comment"># 每个特征有独立的learning rate</span></span><br><span class="line"><span class="keyword">for</span> r <span class="keyword">in</span> range(<span class="number">10000</span>):</span><br><span class="line">    temp_loss = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> m <span class="keyword">in</span> range(<span class="number">36</span>):</span><br><span class="line">        <span class="keyword">for</span> s <span class="keyword">in</span> range(<span class="number">156</span>):</span><br><span class="line">            L = np.dot(w,x[<span class="number">157</span> * m + s].T) - y[<span class="number">157</span> * m + s]</span><br><span class="line">            grad = np.dot(x[<span class="number">157</span> * m + s].T,L)*(<span class="number">2</span>)</span><br><span class="line">            pre_grad += grad**<span class="number">2</span></span><br><span class="line">            ada = np.sqrt(pre_grad)</span><br><span class="line">            w = w - lr * grad/ada</span><br><span class="line">        temp_loss += abs(np.dot(w,x[<span class="number">157</span> * m + <span class="number">156</span>].T) - y[<span class="number">157</span> * m + <span class="number">156</span>])</span><br><span class="line">    print(<span class="string">"%.2f"</span> % (r * <span class="number">100</span> / <span class="number">10000</span>),<span class="string">'% loss:'</span>,<span class="string">"%.4f"</span> % (temp_loss / <span class="number">36</span>))</span><br></pre></td></tr></table></figure>
<blockquote>
<p>保存 weights<br><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">np.save(<span class="string">'model.npy'</span>,w)</span><br></pre></td></tr></table></figure></p>
</blockquote>
<h2 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h2><ul>
<li>加载测试特征数据集(略)</li>
<li>加载label</li>
</ul>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">y = []</span><br><span class="line">rr = <span class="number">0</span></span><br><span class="line"><span class="keyword">with</span> open(<span class="string">'ans.csv'</span>,<span class="string">'r'</span>,encoding = <span class="string">'big5'</span>) <span class="keyword">as</span> ans:</span><br><span class="line">    row = csv.reader(ans,delimiter = <span class="string">','</span>)</span><br><span class="line">    <span class="keyword">for</span> r <span class="keyword">in</span> row:</span><br><span class="line">        <span class="keyword">if</span> rr != <span class="number">0</span>:</span><br><span class="line">            y.append(float(r[<span class="number">1</span>]))</span><br><span class="line">        rr += <span class="number">1</span></span><br><span class="line">y = np.array(y)</span><br></pre></td></tr></table></figure>
<ul>
<li>加载weights</li>
</ul>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">w = np.load(<span class="string">'model.npy'</span>)</span><br></pre></td></tr></table></figure>
<ul>
<li>测试</li>
</ul>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">t = np.dot(x,w)</span><br><span class="line">L = t - y</span><br><span class="line">loss = []</span><br><span class="line">sum = <span class="number">0</span></span><br><span class="line"><span class="keyword">for</span> l <span class="keyword">in</span> L:</span><br><span class="line">    loss.append(abs(l))</span><br><span class="line">    sum += abs(l)</span><br><span class="line">print(sum / len(L))</span><br><span class="line">plt.plot(y,color = <span class="string">"red"</span>,label = <span class="string">'target'</span>)</span><br><span class="line">plt.plot(t,color = <span class="string">"blue"</span>,label = <span class="string">'hypothesis'</span>)</span><br><span class="line">plt.ylabel(<span class="string">'pm 2.5'</span>)</span><br><span class="line">plt.show()</span><br></pre></td></tr></table></figure>
<h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img src="/images/PM25/model_1.png" alt="结论"></p>
<blockquote>
<p>PM2.5 误差 <strong>14.427</strong><br><del>参数太多过拟合</del>没有训练好,卡在了某个地方了,training时候的loss也很高</p>
</blockquote>
<h2 id="再优化"><a href="#再优化" class="headerlink" title="再优化"></a>再优化</h2><blockquote>
<p>只取18个特征中的NMHC、NO2、O3、PM10、PM2.5</p>
</blockquote>
<p><img src="/images/PM25/model_1.png" alt="model_2"></p>
<blockquote>
<p>PM2.5 误差 <strong>8.987</strong><br>一个不错的开头,继续优化</p>
<p>只考虑PM 10和PM 2.5</p>
</blockquote>
<p><img src="/images/PM25/model_3.png" alt="model_3"></p>
<blockquote>
<p>PM2.5 误差 <strong>6.281</strong></p>
</blockquote>
<blockquote>
<p>若在删减特征呢?<br>只考虑 PM2.5 </p>
</blockquote>
<p><img src="/images/PM25/model_4.png" alt="model_4"></p>
<blockquote>
<p>PM2.5 误差 <strong>5.406</strong><br>我服了,之前做的时候是会underfitting导致误差到7.4的,这回倒好更加低了</p>
</blockquote>
<h1 id="使用DNN-对PM2-5进行预测"><a href="#使用DNN-对PM2-5进行预测" class="headerlink" title="使用DNN 对PM2.5进行预测"></a>使用DNN 对PM2.5进行预测</h1><blockquote>
<p>使用的是tensorflow + keras<br>预备工作略</p>
</blockquote>
<h2 id="Feature-Scaling"><a href="#Feature-Scaling" class="headerlink" title="Feature Scaling"></a>Feature Scaling</h2><blockquote>
<p>用了两个不同的Feature Scaling的方法,结果上看差别不大,Standardization更加好一点</p>
</blockquote>
<h3 id="Standardization"><a href="#Standardization" class="headerlink" title="Standardization"></a>Standardization</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">standardization</span><span class="params">(dataMatrix)</span>:</span></span><br><span class="line">    <span class="keyword">if</span> dataMatrix.shape[<span class="number">0</span>] == <span class="number">0</span>:</span><br><span class="line">        <span class="keyword">return</span> dataMatrix</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(dataMatrix.shape[<span class="number">1</span>]):</span><br><span class="line">        sum = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            sum += _x[i]</span><br><span class="line">        mean = sum / dataMatrix.shape[<span class="number">0</span>]</span><br><span class="line">        SD = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            SD += (_x[i] - mean)**<span class="number">2</span></span><br><span class="line">        SD = np.sqrt(SD / dataMatrix.shape[<span class="number">0</span>])</span><br><span class="line">    </span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            _x[i] = (_x[i] - mean) / SD</span><br><span class="line">    <span class="keyword">return</span> dataMatrix</span><br></pre></td></tr></table></figure>
<h3 id="Mean-Normalization"><a href="#Mean-Normalization" class="headerlink" title="Mean Normalization"></a>Mean Normalization</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">meanNormalization</span><span class="params">(dataMatrix)</span>:</span></span><br><span class="line">    <span class="keyword">if</span> dataMatrix.shape[<span class="number">0</span>] == <span class="number">0</span>:</span><br><span class="line">        <span class="keyword">return</span> dataMatrix</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(dataMatrix.shape[<span class="number">1</span>]):</span><br><span class="line">        sum = <span class="number">0</span></span><br><span class="line">        max = <span class="number">0</span></span><br><span class="line">        min = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            sum += data[i]</span><br><span class="line">            <span class="keyword">if</span> data[i] &gt; max:</span><br><span class="line">                max = data[i]</span><br><span class="line">            <span class="keyword">if</span> data[i] &lt; min:</span><br><span class="line">                min = data[i]</span><br><span class="line">        mean = sum / dataMatrix.shape[<span class="number">0</span>]</span><br><span class="line">        <span class="keyword">if</span> (max - min) != <span class="number">0</span>:</span><br><span class="line">            <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">                data[i] = (data[i] - mean) / (max - min)</span><br><span class="line">    <span class="keyword">return</span> dataMatrix</span><br></pre></td></tr></table></figure>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">trainX = standardization(trainX)</span><br><span class="line">testX = standardization(testX)</span><br><span class="line"><span class="comment"># trainX = meanNormalization(trainX)</span></span><br><span class="line"><span class="comment"># testX = meanNormalization(testX)</span></span><br></pre></td></tr></table></figure>
<h2 id="训练-1"><a href="#训练-1" class="headerlink" title="训练"></a>训练</h2><blockquote>
<p>使用output为8的两层layer,激活函数是ReLU(Sigmoid效果更加差)</p>
</blockquote>
<h3 id="建立模型"><a href="#建立模型" class="headerlink" title="建立模型"></a>建立模型</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">model = keras.Sequential([</span><br><span class="line">    keras.layers.Dense(<span class="number">8</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">8</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">1</span>)</span><br><span class="line">    ])</span><br><span class="line">model.compile(loss=<span class="string">"mse"</span>,</span><br><span class="line">        optimizer=tf.train.RMSPropOptimizer(<span class="number">0.001</span>),</span><br><span class="line">        metrics=[<span class="string">'mae'</span>, <span class="string">'mse'</span>])</span><br></pre></td></tr></table></figure>
<h3 id="训练-2"><a href="#训练-2" class="headerlink" title="训练"></a>训练</h3><blockquote>
<p>跑100个epochs基本上没什么变化了<br><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">history = model.fit(trainX, </span><br><span class="line">                    trainY, </span><br><span class="line">                    batch_size = <span class="number">64</span>, </span><br><span class="line">                    epochs = <span class="number">100</span>, </span><br><span class="line">                    validation_split = <span class="number">0.2</span>, </span><br><span class="line">                    verbose=<span class="number">0</span>, </span><br><span class="line">                    callbacks=[PrintDot()])</span><br></pre></td></tr></table></figure></p>
</blockquote>
<blockquote>
<p>训练时候的loss<br><img src="/images/PM25/loss.png" alt="training_loss"></p>
</blockquote>
<h3 id="测试-1"><a href="#测试-1" class="headerlink" title="测试"></a>测试</h3><blockquote>
<p>loss是<strong>7.47</strong></p>
</blockquote>
<p><img src="/images/PM25/dnn.png" alt="test"></p>
<h2 id="再优化-1"><a href="#再优化-1" class="headerlink" title="再优化"></a>再优化</h2><h3 id="不使用Feature-Scaling"><a href="#不使用Feature-Scaling" class="headerlink" title="不使用Feature Scaling"></a>不使用Feature Scaling</h3><blockquote>
<p>训练时候的loss<br><img src="/images/PM25/non_fs.png" alt="nonFS"></p>
</blockquote>
<blockquote>
<p>测试时loss是<strong>5.170</strong><br><img src="/images/PM25/non_fs_rs.png" alt="noFSr"></p>
</blockquote>
<h3 id="增加layers"><a href="#增加layers" class="headerlink" title="增加layers"></a>增加layers</h3><blockquote>
<p>增加多一层layer<br>  训练时loss</p>
</blockquote>
<p><img src="/images/PM25/3layers_loss.png" alt="3layers"></p>
<blockquote>
<p>测试时loss <strong>7.5</strong></p>
</blockquote>
<blockquote>
<p>所以说DNN最优能达到5.17, Linear Model最优5.4<br>不错🎉🎉🎉🎉🎉🎉</p>
</blockquote>
<h1 id="Repo"><a href="#Repo" class="headerlink" title="Repo"></a>Repo</h1><p><a href="https://github.com/VIXNESS/machine-learning-course.git" target="_blank" rel="noopener">VIXNESS/machine-learning-course</a></p>
]]></content>
      <categories>
        <category>machine learning</category>
      </categories>
      <tags>
        <tag>machine learning</tag>
        <tag>linear regression</tag>
      </tags>
  </entry>
  <entry>
    <title>对批判性思维的几点见解</title>
    <url>/2020/04/20/%E5%87%A0%E7%82%B9%E6%89%B9%E5%88%A4%E6%80%A7%E6%80%9D%E7%BB%B4%E7%9A%84%E8%A7%81%E8%A7%A3/</url>
    <content><![CDATA[<h1 id="对批判性思维的几点见解"><a href="#对批判性思维的几点见解" class="headerlink" title="对批判性思维的几点见解"></a>对批判性思维的几点见解</h1><h2 id="论点的结构"><a href="#论点的结构" class="headerlink" title="论点的结构"></a>论点的结构</h2><p>对于接触到的大部分论点/结构/观点, 需要对其内核进行分析.</p><p>大部分情况下, 一个<strong>完整</strong>的观点是由</p><p><strong>论题 —— 论据(原因/理由) —— 结论(结果) —— 立场(价值观)</strong></p><p>组成.</p><p>一般情况下, 论点和论据会显式地给出. 而论题和立场可能需要进行推断.</p><a id="more"></a>




<p>单独给出论题的时候, 抛出的是一个开放性的讨论, 只需要包容地去讨论.</p>
<h2 id="从结论开始切入"><a href="#从结论开始切入" class="headerlink" title="从结论开始切入"></a>从结论开始切入</h2><p>结论和论据出现的先后顺序并不是固定的, 我们必须第一时间抓住其结论, 然后对结论和论据的关系作出分析.</p>
<h3 id="确立对方的结论和论据"><a href="#确立对方的结论和论据" class="headerlink" title="确立对方的结论和论据"></a>确立对方的结论和论据</h3><p>有的时候, 对方会先陈述理由, 再抛出一个结论, 或者反之.</p>
<ul>
<li>对于先讲出理由, 再抛出结论的情况, 我们可以在他陈述完观点后对其进行确认性提问:</li>
</ul>
<blockquote>
<p> “你之所以得出这个结论, 是因为你觉得是….导致的”</p>
</blockquote>
<p>提出类似以上的问题以确认对方的论点和论据.</p>
<ul>
<li>明确陈述中定义</li>
</ul>
<p>有时人们陈述的一些词是关键且模糊、定义不明确的, 或者是很主观的. </p>
<p>我们需要和对方进行一些关键定义的确认. </p>
<p>如果是主观的感受, 我们可以确定一个双方认可的范围.</p>
<blockquote>
<ul>
<li>“食堂的饭菜非常难吃, 这是对学生的不负责任”</li>
</ul>
<p>你所认为的负责是怎样的? —— 明确他所认为的「责任」定义.</p>
<ul>
<li>“这个水果能使得心情变好”</li>
</ul>
<p>你觉得的好心情是指?</p>
</blockquote>
<h3 id="确立对方价值观-立场-目的"><a href="#确立对方价值观-立场-目的" class="headerlink" title="确立对方价值观/立场/目的"></a>确立对方价值观/立场/目的</h3><p>产生争论的一大重要因素是价值观的不同导致的冲突.</p>
<p>当一个人提出他的观点, 我们需要推断他的「价值观」、「立场」和「目的」.</p>
<p>思考是怎样的立场和目的导致他得出的结论.</p>
<p>若发现是价值观的冲突, 我们会求同存异, 而不是进行激烈的争执.</p>
<p>改变他人价值观是一件困难的事情, 我们无需对他人价值观进行强行扭转.</p>
<h3 id="挣脱二元性结论的枷锁"><a href="#挣脱二元性结论的枷锁" class="headerlink" title="挣脱二元性结论的枷锁"></a>挣脱二元性结论的枷锁</h3><p><strong>结论的成立是条件性的:</strong></p>
<p>人们往往喜欢提出「是与非」的结论 —— 不是这样, 就是那样.</p>
<p>我们的第一反应, 不该是「是」或「非」而是对结论进行「目的」、「时间」、「环境」等因素的限定:</p>
<ul>
<li>某些时候结论成立与否.</li>
<li>某些时间成立与否.</li>
<li>出于某某目的是成立的, 出于其他目的是不成立的.</li>
<li>讨论利弊</li>
</ul>
<p><strong>对问题的结论进行扩充:</strong></p>
<ul>
<li>许多问题并不是非黑即白的(折衷性).</li>
<li>是否有其他解决方案(多面性) —— 解决方案是共存的, 并不是唯一的.</li>
</ul>
<h3 id="从原因-立场-目的推出其他结论"><a href="#从原因-立场-目的推出其他结论" class="headerlink" title="从原因/立场/目的推出其他结论"></a>从原因/立场/目的推出其他结论</h3><p>我们可以通过分析支撑结论的论据, 推导出在特定「目的」、「时间」、「环境」下的其他结论.  </p>
<blockquote>
<p>“我们应该关停所有的网吧, 因为网吧使得青少年沉迷网络.”</p>
<p>结论: 我们应该关停网吧.</p>
<p>论据: 网吧会使得青少年沉迷网络.</p>
<p>根据目的扩充: 目的是为了解决青少年沉迷网络 —— 限制青少年的进入, 使用时长以及引导青少年. </p>
</blockquote>
<p><strong>进一步扩充讨论范围.</strong></p>
<p>明确「论题」, 从论题着手, 看看是否有其他的「解决」方法.</p>
<p>从「论题」的角度出发, 将针锋相对的争论化为包容的开放性讨论.</p>
<p>讨论的核心是<strong>解决问题</strong>, 而不是「争论」.</p>
<hr>
<h3 id="分析论据和结论的关系"><a href="#分析论据和结论的关系" class="headerlink" title="分析论据和结论的关系"></a>分析论据和结论的关系</h3><p>完成确立结论和论据的步骤之后, 我们对论据和结论进行关系分析.</p>
<ol>
<li><p>相关性检查: 这个这个论据和结论相关吗? </p>
<ul>
<li><p>人们最容易犯的错误, 将两个无关的东西绑定在一起.</p>
</li>
<li><p>人们很容易犯这种错误, 应首先对其进行检查.</p>
</li>
</ul>
</li>
<li><p>因果性检查: 论据和结论是因果关系么? 是不是其他原因导致的? 是不是和其他原因共同作用导致的? </p>
<ol>
<li>可以使用原因替换的方法, 思考其陈述的原因是否是其结论的直接/主要/决定性的.</li>
<li>想想是否会是其他原因导致的结果, 或其陈述的原因并非是与其结论有因果关系.</li>
</ol>
</li>
</ol>
<hr>
<h2 id="常见的谬误"><a href="#常见的谬误" class="headerlink" title="常见的谬误"></a>常见的谬误</h2><ul>
<li>民调, 大众观点便是正确的 —— 真理既不在多数人手里, 也不在少数人手里, 真理是客观存在, 与人无关.</li>
<li>只有完美解决某个问题的方法才能被接受, 否则就全盘否定它.</li>
<li>扣帽子 —— 因为他做了A这件事, 所以他就是那类人.</li>
<li>推导蔓延 —— 因为他做了A这件事, 所以他也会做B这件事. (禁止作为有效论据)</li>
</ul>
<hr>
<h2 id="谨防真假参半的论证"><a href="#谨防真假参半的论证" class="headerlink" title="谨防真假参半的论证"></a>谨防真假参半的论证</h2><blockquote>
<p>十句话, 九句是真的, 一句是假的.</p>
</blockquote>
<ul>
<li>来源确认.</li>
<li>统计学确认.</li>
<li>专家身份确认.</li>
</ul>
<p>大方向: 无充分证据表明, 则不相信其论点.</p>
<hr>
<h2 id="批判思维对自我的批判"><a href="#批判思维对自我的批判" class="headerlink" title="批判思维对自我的批判"></a>批判思维对自我的批判</h2><ul>
<li>慢思考: 思考多个可能存在的结论, 而不是急于证明自己想到的第一个观点.</li>
<li>尽可能的超越自己的立场和价值观, 思考反对者等其他方面的观点及其理由.</li>
<li>检查自己所用的定义是否精准, 明确自己的目的, 通过目的推演多个候选结论.</li>
<li>通过结论检查自己的论据/原因, 是否是唯一决定结论的: <ul>
<li>很多时候你的因素不是决定性的, 而是导致结论的因素「之一」.</li>
<li>可能根本就不是原因之一.</li>
</ul>
</li>
<li>结论很可能是非二元的:<ul>
<li>折衷的 —— 并不是非黑即白的.</li>
<li>多面的 —— 结论具有多面性, 结论「不唯一」, 更可能是多结论共存的.</li>
</ul>
</li>
<li>慎用类比, 类比会含潜在条件, 与原本的条件不一致.</li>
<li>谨慎的目标性论证: 「先定义一个目标, 再通过各种证据去支撑它」, 这样做会因为其目的性导致论证的片面, 需要慎重, 多思考<em>多面性</em>和<em>反面性</em>.</li>
<li>禁止蔓延式的推导: 因为事件A的发生, 事件B也会发生, 事件C也会发生. (可以考虑可能性, 禁止作为论据).</li>
<li>敢于说出「不知道」: <ul>
<li>对一个事物的不了解, 并不一定个人能力的问题.</li>
<li>而往往是当前无法获得充分的证据来支撑任何论点. </li>
<li>不妄下断言是谨慎的表现.</li>
</ul>
</li>
<li>批判性思维并不是为了和别人针锋相对, 恰恰相反, 要抱着学习和交流的心态进行交流.</li>
</ul>
]]></content>
      <categories>
        <category>others</category>
      </categories>
      <tags>
        <tag>misc</tag>
      </tags>
  </entry>
  <entry>
    <title>DNN 年收入预测</title>
    <url>/2018/12/24/%E5%B9%B4%E6%94%B6%E5%85%A5%E9%A2%84%E6%B5%8B/</url>
    <content><![CDATA[<h1 id="收入预测"><a href="#收入预测" class="headerlink" title="收入预测"></a>收入预测</h1><blockquote>
<p>通过 职位、婚姻情况、学历、家庭角色、种族、性别对其进行年薪的预测🥳🥳</p>
</blockquote><h2 id="Data-Set"><a href="#Data-Set" class="headerlink" title="Data Set"></a>Data Set</h2><p><a href="https://github.com/VIXNESS/machine-learning-course/blob/master/winner/train.csv" target="_blank" rel="noopener">Training Data</a></p><p><a href="https://github.com/VIXNESS/machine-learning-course/blob/master/winner/test.csv" target="_blank" rel="noopener">Testing Data</a></p><h2 id="预备工作"><a href="#预备工作" class="headerlink" title="预备工作"></a>预备工作</h2><h3 id="所需类库"><a href="#所需类库" class="headerlink" title="所需类库"></a>所需类库</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> tensorflow <span class="keyword">as</span> tf</span><br><span class="line"><span class="keyword">from</span> tensorflow <span class="keyword">import</span> keras</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> sys</span><br><span class="line"><span class="keyword">import</span> csv</span><br></pre></td></tr></table></figure><a id="more"></a>



<h3 id="数据装载"><a href="#数据装载" class="headerlink" title="数据装载"></a>数据装载</h3><blockquote>
<p>数据构成: [feature 1, … feature n; &gt;= 50k or &lt; 50k] 😵😵😵<br><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">loadData</span><span class="params">(_x, _y, fileName)</span>:</span></span><br><span class="line">    col_1 = [<span class="string">' State-gov'</span>, </span><br><span class="line">              <span class="string">' Self-emp-not-inc'</span>, </span><br><span class="line">              <span class="string">' Private'</span>, </span><br><span class="line">              <span class="string">' Federal-gov'</span>, </span><br><span class="line">              <span class="string">' Local-gov'</span>, </span><br><span class="line">              <span class="string">' ?'</span>, </span><br><span class="line">              <span class="string">' Self-emp-inc'</span>, </span><br><span class="line">              <span class="string">' Without-pay'</span>, </span><br><span class="line">              <span class="string">' Never-worked'</span>]</span><br><span class="line">    col_3 = [<span class="string">' Bachelors'</span>, </span><br><span class="line">              <span class="string">' HS-grad'</span>, </span><br><span class="line">              <span class="string">' 11th'</span>, </span><br><span class="line">              <span class="string">' Masters'</span>, </span><br><span class="line">              <span class="string">' 9th'</span>, </span><br><span class="line">              <span class="string">' Some-college'</span>, </span><br><span class="line">              <span class="string">' Assoc-acdm'</span>, </span><br><span class="line">              <span class="string">' Assoc-voc'</span>, </span><br><span class="line">              <span class="string">' 7th-8th'</span>, </span><br><span class="line">              <span class="string">' Doctorate'</span>, </span><br><span class="line">              <span class="string">' Prof-school'</span>, </span><br><span class="line">              <span class="string">' 5th-6th'</span>, </span><br><span class="line">              <span class="string">' 10th'</span>, </span><br><span class="line">              <span class="string">' 1st-4th'</span>, </span><br><span class="line">              <span class="string">' Preschool'</span>, </span><br><span class="line">              <span class="string">' 12th'</span>]</span><br><span class="line">    col_5 = [<span class="string">' Never-married'</span>, </span><br><span class="line">              <span class="string">' Married-civ-spouse'</span>, </span><br><span class="line">              <span class="string">' Divorced'</span>, </span><br><span class="line">              <span class="string">' Married-spouse-absent'</span>, </span><br><span class="line">              <span class="string">' Separated'</span>, </span><br><span class="line">              <span class="string">' Married-AF-spouse'</span>, </span><br><span class="line">              <span class="string">' Widowed'</span>]</span><br><span class="line">    col_6 = [<span class="string">' Adm-clerical'</span>, </span><br><span class="line">              <span class="string">' Exec-managerial'</span>, </span><br><span class="line">              <span class="string">' Handlers-cleaners'</span>, </span><br><span class="line">              <span class="string">' Prof-specialty'</span>, </span><br><span class="line">              <span class="string">' Other-service'</span>, </span><br><span class="line">              <span class="string">' Sales'</span>, </span><br><span class="line">              <span class="string">' Craft-repair'</span>, </span><br><span class="line">              <span class="string">' Transport-moving'</span>, </span><br><span class="line">              <span class="string">' Farming-fishing'</span>, </span><br><span class="line">              <span class="string">' Machine-op-inspct'</span>, </span><br><span class="line">              <span class="string">' Tech-support'</span>, </span><br><span class="line">              <span class="string">' ?'</span>, </span><br><span class="line">              <span class="string">' Protective-serv'</span>, </span><br><span class="line">              <span class="string">' Armed-Forces'</span>, </span><br><span class="line">              <span class="string">' Priv-house-serv'</span>]</span><br><span class="line">    col_7 = [<span class="string">' Not-in-family'</span>, </span><br><span class="line">              <span class="string">' Husband'</span>, </span><br><span class="line">              <span class="string">' Wife'</span>, </span><br><span class="line">              <span class="string">' Own-child'</span>, </span><br><span class="line">              <span class="string">' Unmarried'</span>, </span><br><span class="line">              <span class="string">' Other-relative'</span>]</span><br><span class="line">    col_8 = [<span class="string">' White'</span>, </span><br><span class="line">              <span class="string">' Black'</span>, </span><br><span class="line">              <span class="string">' Asian-Pac-Islander'</span>, </span><br><span class="line">              <span class="string">' Amer-Indian-Eskimo'</span>, </span><br><span class="line">              <span class="string">' Other'</span>]</span><br><span class="line">    col_9 = [<span class="string">' Male'</span>, <span class="string">' Female'</span>]</span><br><span class="line">    col_13 = [<span class="string">' United-States'</span>, </span><br><span class="line">              <span class="string">' Cuba'</span>, </span><br><span class="line">              <span class="string">' Jamaica'</span>, </span><br><span class="line">              <span class="string">' India'</span>, </span><br><span class="line">              <span class="string">' ?'</span>, </span><br><span class="line">              <span class="string">' Mexico'</span>, </span><br><span class="line">              <span class="string">' South'</span>, </span><br><span class="line">              <span class="string">' Puerto-Rico'</span>, </span><br><span class="line">              <span class="string">' Honduras'</span>, </span><br><span class="line">              <span class="string">' England'</span>, </span><br><span class="line">              <span class="string">' Canada'</span>, </span><br><span class="line">              <span class="string">' Germany'</span>, </span><br><span class="line">              <span class="string">' Iran'</span>, </span><br><span class="line">              <span class="string">' Philippines'</span>, </span><br><span class="line">              <span class="string">' Italy'</span>, </span><br><span class="line">              <span class="string">' Poland'</span>, </span><br><span class="line">              <span class="string">' Columbia'</span>, </span><br><span class="line">              <span class="string">' Cambodia'</span>, </span><br><span class="line">              <span class="string">' Thailand'</span>, </span><br><span class="line">              <span class="string">' Ecuador'</span>, </span><br><span class="line">              <span class="string">' Laos'</span>, </span><br><span class="line">              <span class="string">' Taiwan'</span>, </span><br><span class="line">              <span class="string">' Haiti'</span>, </span><br><span class="line">              <span class="string">' Portugal'</span>, </span><br><span class="line">              <span class="string">' Dominican-Republic'</span>, </span><br><span class="line">              <span class="string">' El-Salvador'</span>, </span><br><span class="line">              <span class="string">' France'</span>, </span><br><span class="line">              <span class="string">' Guatemala'</span>, </span><br><span class="line">              <span class="string">' China'</span>, </span><br><span class="line">              <span class="string">' Japan'</span>, </span><br><span class="line">              <span class="string">' Yugoslavia'</span>, </span><br><span class="line">              <span class="string">' Peru'</span>, </span><br><span class="line">              <span class="string">' Outlying-US(Guam-USVI-etc)'</span>, </span><br><span class="line">              <span class="string">' Scotland'</span>, </span><br><span class="line">              <span class="string">' Trinadad&amp;Tobago'</span>, </span><br><span class="line">              <span class="string">' Greece'</span>, </span><br><span class="line">              <span class="string">' Nicaragua'</span>, </span><br><span class="line">              <span class="string">' Vietnam'</span>, </span><br><span class="line">              <span class="string">' Hong'</span>, </span><br><span class="line">              <span class="string">' Ireland'</span>, </span><br><span class="line">              <span class="string">' Hungary'</span>, </span><br><span class="line">              <span class="string">' Holand-Netherlands'</span>]</span><br><span class="line">    col_14 = [<span class="string">' &lt;=50K'</span>, <span class="string">' &gt;50K'</span>, <span class="string">' &lt;=50K.'</span>, <span class="string">' &gt;50K.'</span>]</span><br><span class="line">    <span class="keyword">with</span> open(fileName) <span class="keyword">as</span> rawData:</span><br><span class="line">        rows = csv.reader(rawData, delimiter = <span class="string">","</span>)</span><br><span class="line">        <span class="keyword">for</span> r <span class="keyword">in</span> rows:</span><br><span class="line">            <span class="keyword">if</span> len(r) == <span class="number">0</span>:</span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line">            temp = []</span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">15</span>):</span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">0</span>:</span><br><span class="line">                    temp.append(float(r[i]))</span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">1</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_1:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">2</span>:</span><br><span class="line">                    temp.append(float(r[i]))</span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">3</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_3:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">4</span>:</span><br><span class="line">                    temp.append(float(r[i]))</span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">5</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_5:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">6</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_6:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">7</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_7:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">8</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_8:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">9</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_9:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">10</span> <span class="keyword">or</span> i == <span class="number">11</span> <span class="keyword">or</span> i == <span class="number">12</span>:</span><br><span class="line">                    temp.append(float(r[i]))</span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">13</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_13:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            temp.append(float(cnt))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">                <span class="keyword">if</span> i == <span class="number">14</span>:</span><br><span class="line">                    cnt = <span class="number">0</span></span><br><span class="line">                    <span class="keyword">for</span> c <span class="keyword">in</span> col_14:</span><br><span class="line">                        <span class="keyword">if</span> c == r[i]:</span><br><span class="line">                            _y.append(float(cnt % <span class="number">2</span>))</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        cnt += <span class="number">1</span></span><br><span class="line">            _x.append(temp)</span><br></pre></td></tr></table></figure></p>
</blockquote>
<h2 id="模型定义"><a href="#模型定义" class="headerlink" title="模型定义"></a>模型定义</h2><h3 id="Feature-Scaling"><a href="#Feature-Scaling" class="headerlink" title="Feature Scaling"></a>Feature Scaling</h3><blockquote>
<p>4种 Feature Scaling的方法</p>
</blockquote>
<h4 id="Standardization"><a href="#Standardization" class="headerlink" title="Standardization"></a>Standardization</h4><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">standardization</span><span class="params">(dataMatrix)</span>:</span> <span class="comment">#🤩🤩🤩</span></span><br><span class="line">    <span class="keyword">if</span> dataMatrix.shape[<span class="number">0</span>] == <span class="number">0</span>:</span><br><span class="line">        <span class="keyword">return</span> dataMatrix</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(dataMatrix.shape[<span class="number">1</span>]):</span><br><span class="line">        sum = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            sum += _x[i]</span><br><span class="line">        mean = sum / dataMatrix.shape[<span class="number">0</span>]</span><br><span class="line">        SD = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            SD += (_x[i] - mean)**<span class="number">2</span></span><br><span class="line">        SD = np.sqrt(SD / dataMatrix.shape[<span class="number">0</span>])</span><br><span class="line">    </span><br><span class="line">        <span class="keyword">for</span> _x <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            _x[i] = (_x[i] - mean) / SD</span><br><span class="line">    <span class="keyword">return</span> dataMatrix</span><br></pre></td></tr></table></figure>
<h4 id="Mean-Normalization"><a href="#Mean-Normalization" class="headerlink" title="Mean Normalization"></a>Mean Normalization</h4><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">meanNormalization</span><span class="params">(dataMatrix)</span>:</span></span><br><span class="line">    <span class="keyword">if</span> dataMatrix.shape[<span class="number">0</span>] == <span class="number">0</span>:</span><br><span class="line">        <span class="keyword">return</span> dataMatrix</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(dataMatrix.shape[<span class="number">1</span>]):</span><br><span class="line">        sum = <span class="number">0</span></span><br><span class="line">        max = <span class="number">0</span></span><br><span class="line">        min = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            sum += data[i]</span><br><span class="line">            <span class="keyword">if</span> data[i] &gt; max:</span><br><span class="line">                max = data[i]</span><br><span class="line">            <span class="keyword">if</span> data[i] &lt; min:</span><br><span class="line">                min = data[i]</span><br><span class="line">        mean = sum / dataMatrix.shape[<span class="number">0</span>]</span><br><span class="line">        <span class="keyword">if</span> (max - min) != <span class="number">0</span>:</span><br><span class="line">            <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">                data[i] = (data[i] - mean) / (max - min)</span><br><span class="line">    <span class="keyword">return</span> dataMatrix</span><br></pre></td></tr></table></figure>
<h4 id="Rescaling"><a href="#Rescaling" class="headerlink" title="Rescaling"></a>Rescaling</h4><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">rescaling</span><span class="params">(dataMatrix)</span>:</span></span><br><span class="line">    <span class="keyword">if</span> dataMatrix.shape[<span class="number">0</span>] == <span class="number">0</span>:</span><br><span class="line">        <span class="keyword">return</span> dataMatrix</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(dataMatrix.shape[<span class="number">1</span>]):</span><br><span class="line">        max = <span class="number">0</span></span><br><span class="line">        min = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">            <span class="keyword">if</span> data[i] &gt; max:</span><br><span class="line">                max = data[i]</span><br><span class="line">            <span class="keyword">if</span> data[i] &lt; min:</span><br><span class="line">                min = data[i]</span><br><span class="line">        <span class="keyword">if</span> max - min != <span class="number">0</span>:</span><br><span class="line">            <span class="keyword">for</span> data <span class="keyword">in</span> dataMatrix:</span><br><span class="line">                data[i] = (data[i] - min) / (max - min)</span><br><span class="line">    <span class="keyword">return</span> dataMatrix</span><br></pre></td></tr></table></figure>
<blockquote>
<p>使用以上某一种方法对数据进行Feature Scaling<br>经过测试,Standardization的效果最好,其次是 Mean Normalization </p>
</blockquote>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">trainX = []</span><br><span class="line">trainY = []</span><br><span class="line">testX = []</span><br><span class="line">testY = []</span><br><span class="line">loadData(trainX, trainY, <span class="string">'train.csv'</span>)</span><br><span class="line">loadData(testX, testY, <span class="string">'test.csv'</span>)</span><br><span class="line">trainX = np.array(trainX)</span><br><span class="line">trainY = np.array(trainY)</span><br><span class="line">testX = np.array(testX)</span><br><span class="line">testY = np.array(testY)</span><br><span class="line">trainX = standardization(trainX) <span class="comment">#Test accuracy: 0.8504391622392502</span></span><br><span class="line">testX = standardization(testX)</span><br><span class="line"><span class="comment"># trainX = meanNormalization(trainX) #Test accuracy: 0.8516675879786739</span></span><br><span class="line"><span class="comment"># testX = meanNormalization(testX)</span></span><br><span class="line"><span class="comment"># trainX = rescaling(trainX) #Test accuracy: 0.8477366255400303</span></span><br><span class="line"><span class="comment"># testX = rescaling(testX)</span></span><br></pre></td></tr></table></figure>
<h3 id="训练"><a href="#训练" class="headerlink" title="训练"></a>训练</h3><h4 id="设计模型"><a href="#设计模型" class="headerlink" title="设计模型"></a>设计模型</h4><blockquote>
<p>使用两层layers 激活函数 ReLU <del>(就乱设计的)</del> 😅<br>Loss Function 用的是Cross Entropy, 因为使用了ReLU用Square Error会有许多的地方没有梯度,很尴尬🥵</p>
</blockquote>
<figure class="highlight python"><table><tr><td class="code"><pre><span class="line">model = tf.keras.Sequential([</span><br><span class="line">    keras.layers.Dense(<span class="number">28</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">14</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">2</span>,activation=tf.nn.softmax)</span><br><span class="line">])</span><br><span class="line">model.compile(optimizer=tf.train.AdamOptimizer(), </span><br><span class="line">              loss=<span class="string">'sparse_categorical_crossentropy'</span>, </span><br><span class="line">              metrics=[<span class="string">'accuracy'</span>])</span><br></pre></td></tr></table></figure>
<blockquote>
<p>训练时的精准度:</p>
</blockquote>
<p><img src="/images/Winner/acc.png" alt="acc"></p>
<blockquote>
<p>测试精准度:</p>
</blockquote>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">16281/16281 [==============================] - 0s 25us/step</span><br><span class="line">Test loss:  0.3315070253370084   Test accuracy: 0.8466310422863189</span><br></pre></td></tr></table></figure>
<h2 id="负优化"><a href="#负优化" class="headerlink" title="负优化"></a>负优化</h2><h3 id="取消Feature-Scaling"><a href="#取消Feature-Scaling" class="headerlink" title="取消Feature Scaling"></a>取消Feature Scaling</h3><blockquote>
<p>很真实  🥵</p>
</blockquote>
<p><img src="/images/Winner/nonfs.png" alt="non-fs"></p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">16281/16281 [==============================] - 0s 24us/step</span><br><span class="line">Test loss:  12.310577790542826   Test accuracy: 0.23622627602315244</span><br></pre></td></tr></table></figure>
<h3 id="增加layers"><a href="#增加layers" class="headerlink" title="增加layers"></a>增加layers</h3><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">model = tf.keras.Sequential([</span><br><span class="line">    keras.layers.Dense(<span class="number">28</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">28</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">14</span>, activation=tf.nn.relu),</span><br><span class="line">    keras.layers.Dense(<span class="number">2</span>,activation=tf.nn.softmax)</span><br><span class="line">])</span><br></pre></td></tr></table></figure>
<blockquote>
<p>没用🤦‍ 🤦‍ 🤷‍ 🤷‍</p>
</blockquote>
<p><img src="/images/Winner/more.png" alt="more layer"></p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">16281/16281 [==============================] - 0s 26us/step</span><br><span class="line">Test loss:  0.33134235454223626   Test accuracy: 0.8481665745274117</span><br></pre></td></tr></table></figure>
<p>完结🎉🎉🎉</p>
<h2 id="Repo"><a href="#Repo" class="headerlink" title="Repo"></a>Repo</h2><p><a href="https://github.com/VIXNESS/machine-learning-course/tree/master/winner" target="_blank" rel="noopener">&gt;50K?</a></p>
]]></content>
      <categories>
        <category>machine learning</category>
      </categories>
      <tags>
        <tag>machine learning</tag>
        <tag>deep learning</tag>
      </tags>
  </entry>
  <entry>
    <title>Java Note</title>
    <url>/2020/07/31/Java-Note/</url>
    <content><![CDATA[<h1 id="Java-Note"><a href="#Java-Note" class="headerlink" title="Java Note"></a>Java Note</h1><h2 id="I-虚拟机内存布局"><a href="#I-虚拟机内存布局" class="headerlink" title="I 虚拟机内存布局"></a>I 虚拟机内存布局</h2><h3 id="1-JVM运行时-内存布局"><a href="#1-JVM运行时-内存布局" class="headerlink" title="1. JVM运行时 内存布局"></a>1. JVM运行时 内存布局</h3><table>
<thead>
<tr>
<th>区域</th>
<th>作用</th>
<th>线程内存共享</th>
<th>异常</th>
<th>异常原因</th>
</tr>
</thead>
<tbody>
<tr>
<td>方法区</td>
<td>存放类型信息, 常量, 静态变量, JIT代码缓存.</td>
<td>线程共享</td>
<td>OOM</td>
<td>JVM规范规定, 无法满足内存分配时可抛出.</td>
</tr>
<tr>
<td>虚拟机栈</td>
<td>存储局部变量, 操作数栈,  动态链接, 接口等.</td>
<td>线程私有</td>
<td>OOM, SO</td>
<td>SO: 栈深度超出JVM允许范围;<br>OOM: 栈扩展失败导致的内存分配不足.</td>
</tr>
<tr>
<td>本地方法栈</td>
<td>与虚拟机栈功能相同, 但是对象是本地方法(Native Method).</td>
<td>线程私有</td>
<td>OOM, SO</td>
<td>同上.</td>
</tr>
<tr>
<td>堆</td>
<td>绝大部分对象在此存储(除了栈上分配和标量替换的对象).</td>
<td>共享/私有(TLAB)</td>
<td>OOM</td>
<td>堆中无法完成实例分配时抛出.</td>
</tr>
<tr>
<td>PC计数器</td>
<td>下一条指令的地址</td>
<td>线程私有</td>
<td>无</td>
<td>无</td>
</tr>
</tbody>
</table><a id="more"></a>
<blockquote>
<p>OOM: OutOfMemoryError</p>
<p>SO: StackOverFlowError</p>
<p>TLAB: 线程私有分配缓冲区 Thread Local Allocation Buffer</p>
</blockquote>
<h3 id="2-对象内存布局"><a href="#2-对象内存布局" class="headerlink" title="2. 对象内存布局"></a>2. 对象内存布局</h3><p>堆中对象的布局: 对象头 + 数据 + 对其填充.</p>
<p>对象头: 锁信息, GC信息, 偏向信息.</p>
<h3 id="3-对象访问"><a href="#3-对象访问" class="headerlink" title="3. 对象访问"></a>3. 对象访问</h3><h4 id="3-1-句柄访问"><a href="#3-1-句柄访问" class="headerlink" title="3.1 句柄访问"></a>3.1 句柄访问</h4><p><img src="/images/Java_Note/0.png" alt="句柄"> </p>
<h4 id="3-2-指针访问"><a href="#3-2-指针访问" class="headerlink" title="3.2 指针访问"></a>3.2 指针访问</h4><p><img src="/images/Java_Note/1.png" alt="指针"> </p>
<h2 id="II-垃圾回收策略"><a href="#II-垃圾回收策略" class="headerlink" title="II 垃圾回收策略"></a>II 垃圾回收策略</h2><h3 id="1-回收理论"><a href="#1-回收理论" class="headerlink" title="1. 回收理论"></a>1. 回收理论</h3><h4 id="对象存活判定"><a href="#对象存活判定" class="headerlink" title="对象存活判定"></a>对象存活判定</h4><p>应回收对象的判断: 可达性分析.</p>
<blockquote>
<p>可达性分析:</p>
<p>以一些可作为GC Root的对象为根的一棵由引用关系建立起来的树, 当树根死去, 整个棵树上的对象都将不可达.</p>
<p>什么对象可作为GC Root:</p>
<ul>
<li>虚拟栈中引用的对象</li>
<li>方法区中静态变量所引用的变量</li>
<li>常量引用的对象</li>
<li>JVM内部引用(例如一些Class对象, 异常对象, 类加载器)</li>
<li>synchronized持有的对象</li>
</ul>
</blockquote>
<h4 id="分代假说"><a href="#分代假说" class="headerlink" title="分代假说"></a>分代假说</h4><ul>
<li>大多数对象都是朝生夕灭的.</li>
<li>几次GC之后仍然存活的对象是不朽的.</li>
<li>跨代引用(新生代, 老年代之间的引用) 是极少的.</li>
</ul>
<p>所以将需要GC的区域分为Eden区和Survivors区.</p>
<p>新建立的对象放在Eden区.</p>
<p>经过几代GC仍然存活的对象放入Survivors区.</p>
<h4 id="回收算法"><a href="#回收算法" class="headerlink" title="回收算法"></a>回收算法</h4><ol>
<li>标记 - 清除:  (产生大量碎片)<ol>
<li>先标记需要回收的对象.</li>
<li>清除对象.</li>
</ol>
</li>
<li>标记 - 复制: (将区域划分为两个区域, 回收时从一个区域把存活对象复制到另一个区域, 并且紧密排列)<ol>
<li>先标记要回收的对象.</li>
<li>复制存活到对象到另一个半区.</li>
</ol>
</li>
<li>标记 - 整理: (移动每次都存活的老年代代价是非常高的, 而且需要暂停用户线程)<ol>
<li>先标记要回收的对象.</li>
<li>将存活的对象进行整理, 使其紧密排列.</li>
</ol>
</li>
</ol>
<h3 id="2-回收时机"><a href="#2-回收时机" class="headerlink" title="2. 回收时机"></a>2. 回收时机</h3><p>强引用: 一般的声明的对象, 只要关系存在, 不会被回收.</p>
<p>软引用: 非必须对象,  SoftReference,  在内存溢出之前优先回收.</p>
<p>弱引用: 非必须对象, WeakReference, 一旦触发GC就立刻回收.</p>
<p>虚引用: 不影响对象的声明走起, PhantomReference, 无法取得实例数据, 只能用于接受GC回收的通知.</p>
<p>Minor GC触发:  实例分配失败, Eden区满了.</p>
<p>Full GC触发:  <code>System.gc()</code> , heap dump,  Survivor区的大小无法满足Minor GC后的对象.</p>
<blockquote>
<p>Minor GC: 只回收新生代</p>
</blockquote>
<h3 id="3-回收器例举"><a href="#3-回收器例举" class="headerlink" title="3. 回收器例举"></a>3. 回收器例举</h3><h4 id="3-1-Serial-ParNew"><a href="#3-1-Serial-ParNew" class="headerlink" title="3.1 Serial / ParNew"></a>3.1 Serial / ParNew</h4><p>新生代: 标记 - 复制</p>
<p>老年代: 标记 - 清理</p>
<h4 id="3-2-CMS-注重响应时间-停顿时间短"><a href="#3-2-CMS-注重响应时间-停顿时间短" class="headerlink" title="3.2 CMS(注重响应时间, 停顿时间短)"></a>3.2 CMS(注重响应时间, 停顿时间短)</h4><p>使用标记 - 清除算法. </p>
<ol>
<li>初始标记: 标记GC Root直接关联的对象. (需要Stop The World)</li>
<li>并发标记: 从GC Root开始遍历对象图. </li>
<li>重新标记: 修正并发标记时对象的变动. (需要Stop The World)</li>
<li>并发清除. <strong>(Major GC)</strong></li>
</ol>
<blockquote>
<ul>
<li>CMS无法处理<strong>浮动垃圾</strong>,  因而引发一次Full GC. (浮动垃圾是指CMS在并发清理的时候, 伴随着产生的新垃圾对象.)</li>
<li>CMS收集完成后, 内存空间有碎片存在.</li>
</ul>
</blockquote>
<h4 id="3-3-G1-Mixed-GC"><a href="#3-3-G1-Mixed-GC" class="headerlink" title="3.3 G1 (Mixed GC)"></a>3.3 G1 (Mixed GC)</h4><blockquote>
<p> G1将空间分为大小不一的Region, 每个Region可作为Eden, Survivor空间.</p>
</blockquote>
<p>GC步骤:</p>
<ol>
<li>初始标记: 同CMS(需要Stop The World)</li>
<li>并发标记: 同CMS</li>
<li>最终标记: 修正并发标记时对象的变动. (需要Stop The World)</li>
<li>筛选回收: 根据Region的统计数据对Region进行回收</li>
</ol>
<h4 id="3-4-ZGC"><a href="#3-4-ZGC" class="headerlink" title="3.4 ZGC"></a>3.4 ZGC</h4><blockquote>
<p>与G1一样, 也是使用大小不同的Region拆分回收区域.</p>
<p>并且使用染色指针技术.(具体原理略)</p>
</blockquote>
<p>GC步骤:</p>
<ol>
<li>并发标记: 同CMS</li>
<li>并发预备重分配: 根据查询条件统计出需要回收的Region.</li>
<li>并发重分配: 把集中存活的对象复制到新Region上.</li>
<li>并发重映射: 修正堆中指向旧对象的引用.</li>
</ol>
<h2 id="III-类加载机制"><a href="#III-类加载机制" class="headerlink" title="III 类加载机制"></a>III 类加载机制</h2><h3 id="1-类加载步骤"><a href="#1-类加载步骤" class="headerlink" title="1. 类加载步骤"></a>1. 类加载步骤</h3><ol>
<li><p>加载: 从不同的源头读取二进制字节流, 并且创建Class对象.</p>
</li>
<li><p>验证: 验证文件.</p>
</li>
<li><p>准备: 为静态变量设定初始值(不是赋语句中定义的值, 而是各类型的默认值, 例如int的默认值是0). 初始在方法区.</p>
</li>
<li><p>解析: 将常量池中的符号引用替换为直接引用. (包括类、接口的解析, 字段的解析, 方法的解析, 接口方法的解析)</p>
</li>
<li><p>初始化: 初始化变量, 赋值实际的值给变量. 执行类构造器\&lt;clinit>().</p>
</li>
</ol>
<h3 id="2-类加载器"><a href="#2-类加载器" class="headerlink" title="2. 类加载器"></a>2. 类加载器</h3><blockquote>
<p>不同类加载器加载的同一个类在JVM中不视为同一个类.</p>
</blockquote>
<h4 id="类加载机制"><a href="#类加载机制" class="headerlink" title="类加载机制"></a>类加载机制</h4><p><strong>双亲委派模型</strong> (递归加载)</p>
<ol>
<li>遇到要加载的类优先交由父类加载.</li>
<li>父类无法加载的类再由子类加载.</li>
</ol>
<p><strong>非双亲委派模型</strong></p>
<ol>
<li>线程上下文加载器: 可由父类请求子类类加载器加载.</li>
<li>模块化系统:  若可以找到对应的模块类加载器, 优先派发给其加载. 否则交由父类加载.</li>
</ol>
<h3 id="3-反射系统"><a href="#3-反射系统" class="headerlink" title="3. 反射系统"></a>3. 反射系统</h3><p>反射系统主要是基于Class对象的操作.</p>
<p>当类加载器从不同渠道加载完Java类后, 都会形成这个类加载器下一一对应的Class对象(当然, 不同的类加载器加载的同一个类不是同一个Class对象).</p>
<h2 id="IV-多线程并发"><a href="#IV-多线程并发" class="headerlink" title="IV 多线程并发"></a>IV 多线程并发</h2><h3 id="1-内存模型"><a href="#1-内存模型" class="headerlink" title="1. 内存模型"></a>1. 内存模型</h3><p>JVM主要解决各个线程的工作内存与主存之间的数据一致性问题.</p>
<p><img src="/images/Java_Note/mermaid-diagram-20200731122443.png" alt="内存"> </p>
<h4 id="一致性协议具体内容"><a href="#一致性协议具体内容" class="headerlink" title="一致性协议具体内容"></a>一致性协议具体内容</h4><ul>
<li>read - load: 两条指令必须按<strong><em>顺序</em></strong>使用, 且必须<strong><em>成对</em></strong>出现, read作用于<strong>主存</strong>, load作用于<strong>工作内存</strong>. 目的是为了把<strong>主存</strong>中的变量读入线程<strong>工作内存</strong>变为私有副本.</li>
<li>store - write: 两条指令必须按<strong><em>顺序</em></strong>使用, 且必须<strong><em>成对</em></strong>出现, store作用于<strong>工作内存</strong>, write作用于<strong>主存</strong>. 目的是把<strong>工作内存</strong>中的放入<strong>主存</strong>.</li>
<li>use/assign:  作用于<strong>工作内存</strong>, 将<strong>工作内存</strong>中的数据传输给执行引擎/将执行引擎得到的结果赋值给<strong>工作内存</strong>.</li>
<li>lock/unlock: 作用于<strong>主存</strong>, 将一个变量锁定🔒/解锁🔓为线程独占的状态. <ul>
<li>lock会导致<strong>工作内存</strong>清空, 且需要进行load(主存给出)或者assign(计算引擎给出)对变量进行赋值.</li>
<li>一个变量只能同时被一个线程lock, 但是一个变量可以被同一个线程lock多次, 不过必须有相同数量的unlock.</li>
<li>unlock之前必须有store - write 且之前必须被lock锁定.</li>
</ul>
</li>
</ul>
<h4 id="volatile-关键字"><a href="#volatile-关键字" class="headerlink" title="volatile 关键字"></a>volatile 关键字</h4><p>提供了有序性: 禁止内存重排序优化*. </p>
<p>提供了可见性: 原理是直接将处理机中的Cache无效化, 使得所有线程不得不从主存里读取数据.</p>
<p><strong>不保证原子性.</strong></p>
<h3 id="2-线程实现"><a href="#2-线程实现" class="headerlink" title="2. 线程实现"></a>2. 线程实现</h3><p>Thread对象中的run()是Thread对象将执行代码逻辑的部分, 若直接调用则与普通的一个实现了run()方法的类无差别.</p>
<p>Thread对象中的start()方法作用是处理一套进程准备工作(包括调用本地系统调用创建线程). 线程创建完毕后, 将run()中写好的代码注入到该线程中执行该线程.</p>
<h3 id="3-线程安全"><a href="#3-线程安全" class="headerlink" title="3. 线程安全"></a>3. 线程安全</h3><p>定义: 多个线程同时访问一个对象时, 若不考虑线程调度, 交替, 同步手段和任何协调操作, 单次调用该对象可以获得正确结果, 称之为线程安全.</p>
<blockquote>
<p><strong>线程安全的类就不需要添加同步手段了么?</strong></p>
<p>非也.</p>
<p>大多线程安全的容器都是在方法处加入synchronized关键字, 而非在变量处.</p>
<p>这就导致两个线程同时使用两个不同的方法对统一对象进行修改, 仍然会出现线程不安全.</p>
</blockquote>
<h4 id="实现线程安全编程"><a href="#实现线程安全编程" class="headerlink" title="实现线程安全编程"></a>实现线程安全编程</h4><blockquote>
<p>线程安全的目的是线程之间必须要同步.</p>
</blockquote>
<p><strong>1.  无同步</strong></p>
<ul>
<li>使用<strong>纯代码(可重入代码)</strong> , 纯代码具有一些特性, 例如:    <ul>
<li>一些不依赖全局变量,共享变量, 不实用公共资源, 所以状态量均由参数传入.</li>
<li>结果是可预测的, 相同的输入, 结果也是相同的.</li>
</ul>
</li>
<li>使用线程本地存储TLS —— Thread Local Storage.</li>
</ul>
<p><strong>2. 非阻塞的同步</strong></p>
<blockquote>
<p> 利用 Test-and-Set, CAS等硬件指令, 实现的轻量级锁,乐观锁. </p>
</blockquote>
<p>其思想是: </p>
<ul>
<li>直接在CPU时间里测试目标是否被征用, 若没有被征用直接执行.</li>
<li>若被征用, 则直接在CPU时间内等待, 不去阻塞线程.避免了阻塞/唤醒的线程切换开销.</li>
</ul>
<p>Java中带有concrrent的类常常使用此类方法进行同步.</p>
<p><strong>3. 阻塞的同步</strong></p>
<blockquote>
<p>使用synchronized和ReentrantLock等实现的重量级锁, 悲观锁.</p>
</blockquote>
<p>在内核态, 用户态之间切换, 以阻塞/唤醒线程.</p>
<h3 id="4-锁"><a href="#4-锁" class="headerlink" title="4. 锁"></a>4. 锁</h3><h4 id="4-1-可重入锁-java-util-concurrent-locks-ReentrantLock-vs-synchronized-关键字"><a href="#4-1-可重入锁-java-util-concurrent-locks-ReentrantLock-vs-synchronized-关键字" class="headerlink" title="4.1 可重入锁  java.util.concurrent.locks.ReentrantLock vs. synchronized 关键字"></a>4.1 可重入锁  java.util.concurrent.locks.ReentrantLock vs. synchronized 关键字</h4><blockquote>
<p>可重入: 是指被中断的函数在调度回来之后可以继续执行.</p>
<p>不可重入: 是指被中断后返回后的操作已经和不中断的情况不等效了. 例如对公共变量进行操作, 突然被中断.</p>
</blockquote>
<p>可重入锁: 同一个线程对一段代码加锁之后, 中断后可再次获得该锁.</p>
<p>ReentrantLock 与 synchronized的区别在于:</p>
<ul>
<li>公平锁方面: ReentrantLock可选择为公平锁模式, 而synchronized只能为非公平锁.</li>
<li>代码方面: ReentrantLock加锁解锁不受代码块限制, 可以在一个方法中加锁, 另一个方法中解锁; synchronized只能限制在一个代码块中.</li>
<li>等待中断方面: ReentrantLock可在线程长期不释放资源的时候, 等待资源的线程放弃等待.</li>
<li>唤醒条件方面: ReentrantLock可以绑定多个条件.</li>
</ul>
<p><strong>synchronized 实现原理:</strong></p>
<p>在synchronized包裹的代码块的前后插入管程指令.</p>
<p><strong>ReentrantLock 实现原理: </strong></p>
<blockquote>
<p>数据结构: CLH队列 —— 一个基于双向链表的FIFO队列, 它和普通的队列的区别在于加入了对队列中元素的标记, 用于标记排队线程的状态.</p>
<p>PROPAGATE 节点: 唤醒它, 它还会传播式唤醒它后面的节点, 用于实现共享锁.</p>
<p>SIGNAL 节点: 等待前方的节点唤醒自己.</p>
<p>CONDITION 节点: 表示排序线程在<strong>条件队列</strong>中.</p>
<p>CANCELLED 节点: 该线程被取消.</p>
<p>0: 新进线程的节点.</p>
</blockquote>
<ul>
<li>ReentrantLock 使用 AbstractQueuedSynchronizer 来存储两个队列: 条件队列和等待队列.</li>
<li>条件队列和等待队列中存放的都是排队的线程.</li>
</ul>
<blockquote>
<p>条件队列与等待队列的作用相似, 都是存放排队的线程. </p>
<p>只是一些条件唤醒的线程尚未满足条件, 需要在条件队列中等待, 当满足条件后, 将其加入等待队列, 进行等待.</p>
<p>进入等待队列的线程, 就开始等待获取锁.</p>
</blockquote>
<p><strong>共享锁的实现</strong></p>
<p>节点带PROPAGATE标志插入等待队列, 当该节点被唤醒, 会传播唤醒后续节点.</p>
<p><strong>可重入的实现</strong></p>
<p>检查该持有锁的线程是不是当前线程, 若是则返回该锁.</p>
<p><strong>公平/非公平的实现</strong></p>
<p>公平: 先进入AQS排队.</p>
<p>非公平锁: 基于CAS检测当前是否有线程持有锁, 若没有, 进而直接插队.</p>
<h4 id="4-2-读写锁-Java-util-concurrent-locks-ReentrantReadWriteLock"><a href="#4-2-读写锁-Java-util-concurrent-locks-ReentrantReadWriteLock" class="headerlink" title="4.2 读写锁 Java.util.concurrent.locks.ReentrantReadWriteLock"></a>4.2 读写锁 Java.util.concurrent.locks.ReentrantReadWriteLock</h4><ul>
<li>多个读操作不会阻塞.</li>
<li>一个写操作阻塞所有读操作和所有写操作.</li>
</ul>
<h4 id="4-3-轻量锁"><a href="#4-3-轻量锁" class="headerlink" title="4.3 轻量锁"></a>4.3 轻量锁</h4><ul>
<li>若检测到无锁定, JVM给要加锁的对象创建一个Lock Record 存储对象头(Mark Word)的拷贝.</li>
<li>JVM使用CAS操作更新对象头至Lock Record说明该线程持有了这个锁.</li>
<li>若失败, 说明当前有线程占用该对象.</li>
<li>解锁仍使用CAS操作.</li>
</ul>
<blockquote>
<p>轻量级锁只适合至多2个线程的竞争, 多于2个将膨胀为重量级锁.</p>
</blockquote>
<h4 id="4-4-偏向锁"><a href="#4-4-偏向锁" class="headerlink" title="4.4 偏向锁"></a>4.4 偏向锁</h4><ul>
<li>该锁将偏向于第一个获得他的线程, 若锁一直没有其他线程获取, 则持有偏向锁的线程永不同步, 直接消除同步.</li>
<li>适用于只有一个线程访问同步块.</li>
</ul>
<h4 id="4-5-自旋锁"><a href="#4-5-自旋锁" class="headerlink" title="4.5 自旋锁"></a>4.5 自旋锁</h4><p>自旋锁: 在对象被锁定的时候, 让下一个请求者稍等一下 —— 即执行一下忙循环不放弃CPU.</p>
<p>自适应自旋锁:  利用算法判定是否使用自旋锁, 或者取消自旋锁.</p>
<blockquote>
<p>算法思想:</p>
<p>如果该线程刚刚成功获取到了自旋锁, 那么我们相信他这次也能获得自旋锁.</p>
<p>若该线程很少成功获得自旋锁, 那么我们就取消自旋锁.</p>
</blockquote>
<h4 id="4-6-锁消除"><a href="#4-6-锁消除" class="headerlink" title="4.6 锁消除"></a>4.6 锁消除</h4><p>编译器运行时通过<strong>逃逸分析</strong>, 判定数据不会产生<strong>线程逃逸</strong>后进行锁消除.</p>
<h4 id="4-7-锁粗化"><a href="#4-7-锁粗化" class="headerlink" title="4.7 锁粗化"></a>4.7 锁粗化</h4><p>将连续的多个对同一个对象的加锁解锁操作粗化为对一个代码块的一次性加锁解锁.</p>
<h4 id="4-8-公平锁-非公平锁"><a href="#4-8-公平锁-非公平锁" class="headerlink" title="4.8 公平锁/非公平锁"></a>4.8 公平锁/非公平锁</h4><blockquote>
<p>定义: </p>
<ul>
<li><p>公平锁 —— 按照请求锁的排队顺序分发锁.</p>
</li>
<li><p>非公平锁 —— 随机选取正在等待锁的线程, 给予其锁.</p>
</li>
</ul>
</blockquote>
<p>synchronized 关键字是非公平锁.</p>
<p>ReentrantLock 默认非公平锁, 但是可以设置为公平锁.</p>
<h4 id="4-9-双锁检测-DCL-Double-Check-Lock"><a href="#4-9-双锁检测-DCL-Double-Check-Lock" class="headerlink" title="4.9 双锁检测 DCL(Double Check Lock)"></a>4.9 双锁检测 DCL(Double Check Lock)</h4><p>原理:</p>
<ol>
<li>先检测标志, 相关判定条件(例如持有临界资源的时候放置的标志)</li>
<li>检测成功, 则进入临界区, 否则不进入.</li>
</ol>
<p>优点:</p>
<ul>
<li>可以在条件不成熟的时候不持有临界资源, 减少消耗.</li>
</ul>
<p>缺点:</p>
<ul>
<li>未使用<code>volatile</code> 的标志, 会导致从指令重排序, 导致线程不安全. 解决方法就是对需要检测的标识进行<code>volatile</code>标记.</li>
</ul>
<h4 id="4-10-Atomic类的实现原理"><a href="#4-10-Atomic类的实现原理" class="headerlink" title="4.10 Atomic类的实现原理"></a>4.10 Atomic类的实现原理</h4><p><strong>CAS: check and swap</strong></p>
<p>原理: 一条CPU指令, 具有原子性.</p>
<p>给出一个期望值, 再给出一个要替换值.</p>
<p>若测试目标符合期望值, 则将目标的值换成要替换的值.</p>
<p><strong>volatile &amp; CAS</strong></p>
<ul>
<li><p>volatile保证可见性和有序性.</p>
</li>
<li><p>CAS保证原子性.</p>
</li>
</ul>
<p>CAS的缺点:</p>
<ol>
<li>ABA问题 —— 添加版本号字段, 判定时还要判断是否版本一致.</li>
<li>CPU循环开销.</li>
<li>只能操作一个变量.</li>
</ol>
<h3 id="5-有序性-可见性-原子性"><a href="#5-有序性-可见性-原子性" class="headerlink" title="5. 有序性, 可见性, 原子性"></a>5. 有序性, 可见性, 原子性</h3><blockquote>
<p>因为没有原子性而引发并发错误的操作包括但不限于:</p>
<ul>
<li>check-then-act</li>
<li>read-modify-write</li>
</ul>
</blockquote>
<blockquote>
<p>重排序是指CPU会将无关的两个变量调序, 以达到更高的性能. 但是这往往会导致非期望的顺序.</p>
<p>禁止重排序则是将<strong>内存屏障</strong>插入代码中, 将重排序禁用.</p>
</blockquote>
<h4 id="5-1-volatitle"><a href="#5-1-volatitle" class="headerlink" title="5.1 volatitle"></a>5.1 volatitle</h4><p>volatile具有有序性: 内存屏障; 也具有可见性: 使得Cache失效.</p>
<p>volatile 变量的读操作的确很接近于读取普通变量的速度, 但是写操作需要插入大量的内存屏障, 会产生开销.</p>
<p>但大多数场景性能优于synchronized.</p>
<h4 id="5-2-synchronized"><a href="#5-2-synchronized" class="headerlink" title="5.2 synchronized"></a>5.2 synchronized</h4><p>synchronized具有: 有序性, 可见性, 原子性.</p>
<h4 id="5-3-final"><a href="#5-3-final" class="headerlink" title="5.3 final"></a>5.3 final</h4><p>不可变对象线程安全.</p>
<h3 id="6-long-amp-double"><a href="#6-long-amp-double" class="headerlink" title="6. long&amp;double"></a>6. long&amp;double</h3><p>long &amp; double 在 32-bit JVM 下是非原子性协议的, 因为JVM允许2Byte的数据类型分两次读写.</p>
<p>但是64-bit JVM不会出现这种非原子协议的情况.</p>
<h3 id="6-线程池"><a href="#6-线程池" class="headerlink" title="6. 线程池"></a>6. 线程池</h3><p><strong>线程池的作用:</strong> </p>
<ol>
<li>减少频繁申请/销毁线程的资源.</li>
<li>减少无限制申请带来的消耗资源的风险.</li>
</ol>
<p><strong>ThreadPoolExecutor的思想:</strong></p>
<ul>
<li><p>首先将任务和线程分离,  任务放在阻塞队列或者立即执行, 线程放在线程池.</p>
</li>
<li><p>线程池和任务队列用一个线程分配器衔接.</p>
</li>
<li><p>线程分配器拿到任务后从线程池中获取线程, 然后再从阻塞队列中拿(也可以选取需要直接执行的任务) 进行执行.</p>
</li>
<li><p>线程池部分则负责线程的创建和回收, 与任务解耦.</p>
</li>
</ul>
<p><strong>任务调度思想</strong></p>
<p>线程小于核心数, 线程池正在运行, 即可执行.</p>
<p>否则加入阻塞队列, 若队列满了, 就拒绝任务.</p>
<p>线程池要是未运行, 也拒绝任务.</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Callable&lt;V&gt;;<span class="comment">// 带返回的, 抛异常的Runnable</span></span><br><span class="line">Future&lt;V&gt;;</span><br></pre></td></tr></table></figure>
<h3 id="线程同步"><a href="#线程同步" class="headerlink" title="线程同步"></a>线程同步</h3><h4 id="notify-notifyAll-wait"><a href="#notify-notifyAll-wait" class="headerlink" title="notify()/notifyAll() + wait()"></a>notify()/notifyAll() + wait()</h4><p>缺点: </p>
<ul>
<li>过早唤醒: 唤醒条件未成立的情况下, 被提前唤醒.</li>
<li>信号丢失: .wait()被暂停(先notify(), 之后才wait()), 导致没有收到notify进而导致无限等待.</li>
<li>欺骗性唤醒: 没有其他线程执行而导致唤醒(小概率).</li>
<li>notifyAll()可导致多个线程的上下文切换, 开销增大.</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Object object = <span class="keyword">new</span> Object();</span><br><span class="line">AtomicBoolean flag = <span class="keyword">new</span> AtomicBoolean(<span class="keyword">true</span>);</span><br><span class="line"><span class="keyword">new</span> Thread(()-&gt;&#123;</span><br><span class="line">    <span class="keyword">synchronized</span> (object) &#123;</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            Thread.sleep(<span class="number">1000</span>);</span><br><span class="line">            <span class="keyword">while</span>(flag.get())&#123; <span class="comment">//保护条件</span></span><br><span class="line">                object.wait();</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">            e.printStackTrace();</span><br><span class="line">        &#125;</span><br><span class="line">        System.out.println(Thread.currentThread().getName() + <span class="string">" is awake."</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;).start();</span><br><span class="line"></span><br><span class="line"><span class="keyword">new</span> Thread(() -&gt; &#123;</span><br><span class="line">    <span class="keyword">synchronized</span> (object) &#123;</span><br><span class="line">        System.out.println(<span class="string">"Prepare for notifying"</span>);</span><br><span class="line">        flag.getAndSet(<span class="keyword">false</span>);</span><br><span class="line">        object.notifyAll();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;).start();</span><br></pre></td></tr></table></figure>
<h4 id="join"><a href="#join" class="headerlink" title="join()"></a>join()</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Thread th1 = <span class="keyword">new</span> Thread(() -&gt; &#123;</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        Thread.sleep(<span class="number">1000</span>);</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;);</span><br><span class="line"></span><br><span class="line"><span class="keyword">new</span> Thread(()-&gt;&#123;</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        th1.join();<span class="comment">//后续线程</span></span><br><span class="line">        System.out.println(<span class="string">"awake"</span>);</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;).start();</span><br><span class="line">th1.start();</span><br></pre></td></tr></table></figure>
<h4 id="Condition-接口"><a href="#Condition-接口" class="headerlink" title="Condition 接口"></a>Condition 接口</h4><blockquote>
<p>可以解决过早唤醒的问题.</p>
</blockquote>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> Lock lock = <span class="keyword">new</span> ReentrantLock(); <span class="comment">//不可变对象可以避免许多问题</span></span><br><span class="line"><span class="keyword">final</span> Condition condition = lock.newCondition(); <span class="comment">//Condition由lock给出</span></span><br><span class="line">AtomicBoolean flag = <span class="keyword">new</span> AtomicBoolean(<span class="keyword">true</span>);</span><br><span class="line">Runnable runnable = ()-&gt;&#123;</span><br><span class="line">    lock.lock(); <span class="comment">//使用condition之前要上锁, 以防欺骗性唤醒, 提前唤醒, 信号丢失</span></span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        <span class="keyword">while</span>(flag.get()) &#123;</span><br><span class="line">            condition.await();</span><br><span class="line">        &#125;</span><br><span class="line">        System.out.println(Thread.currentThread().getName() + <span class="string">" is awake"</span>);</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125;<span class="keyword">finally</span> &#123;</span><br><span class="line">        lock.unlock();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line"><span class="keyword">new</span> Thread(runnable).start();</span><br><span class="line"><span class="keyword">new</span> Thread(runnable).start();</span><br><span class="line"><span class="keyword">new</span> Thread(()-&gt;&#123;</span><br><span class="line">    lock.lock(); <span class="comment">//使用condition之前要上锁</span></span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        Thread.sleep(<span class="number">1000</span>);</span><br><span class="line">        flag.set(<span class="keyword">false</span>);</span><br><span class="line">        condition.signalAll();</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">       lock.unlock();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;).start();</span><br></pre></td></tr></table></figure>
<h4 id="CountDownLatch"><a href="#CountDownLatch" class="headerlink" title="CountDownLatch"></a>CountDownLatch</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> CountDownLatch countDownLatch = <span class="keyword">new</span> CountDownLatch(<span class="number">3</span>);</span><br><span class="line">Runnable runnable = () -&gt; &#123;</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        Thread.sleep(<span class="number">500</span>);</span><br><span class="line">        System.out.println(countDownLatch.getCount());</span><br><span class="line">        countDownLatch.countDown();</span><br><span class="line">        Thread.sleep(<span class="number">500</span>);</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">&#125;;</span><br><span class="line"></span><br><span class="line">IntStream.range(<span class="number">0</span>,<span class="number">10</span>).forEach(i -&gt; &#123;  <span class="comment">// 过多的countDown 只会让 count永远呆在0</span></span><br><span class="line">    <span class="keyword">new</span> Thread(runnable).start();</span><br><span class="line">&#125;);</span><br><span class="line"></span><br><span class="line"><span class="keyword">new</span> Thread(()-&gt;&#123;</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        countDownLatch.await();</span><br><span class="line">    &#125; <span class="keyword">catch</span> (InterruptedException e) &#123;</span><br><span class="line">        e.printStackTrace();</span><br><span class="line">    &#125;</span><br><span class="line">    System.out.println(<span class="string">"finish"</span>);</span><br><span class="line">&#125;).start();</span><br></pre></td></tr></table></figure>
<h4 id="CyclicBarrier"><a href="#CyclicBarrier" class="headerlink" title="CyclicBarrier"></a>CyclicBarrier</h4><blockquote>
<p>原理:</p>
<ul>
<li>线程间阻塞使用ReentrantLock</li>
<li>线程间同步使用Condition</li>
</ul>
</blockquote>
<p>用法:</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Runable lastRunnable = () -&gt; &#123;...&#125;;</span><br><span class="line"><span class="keyword">final</span> CyclicBarrier cyclicBarrier = <span class="keyword">new</span> CyclicBarrier(n, lastRunnable); <span class="comment">// n=调用.await()的线程个数</span></span><br><span class="line">Runable invokeAwaitRunnable = () -&gt; &#123;</span><br><span class="line">  <span class="comment">//...</span></span><br><span class="line">  cyclicBarrier.await();</span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">new</span> Thread(invokeAwaitRunnable).start();<span class="comment">//await()的调用数量达到n, 则cyclicBarrier自动启动lastRunnable, 无需new Thread(lastRunnable).start();</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// 简单同步:</span></span><br><span class="line"><span class="keyword">final</span> CyclicBarrier cyclicBarrierSimple = <span class="keyword">new</span> CyclicBarrier(n); </span><br><span class="line">Runnable run = ()-&gt; &#123;</span><br><span class="line">  <span class="comment">//所有线程都调用await(), 在cyclicBarrierSimple耗尽之前均阻塞, cyclicBarrierSimple一旦耗尽, 则所有调用await()的线程均唤醒.</span></span><br><span class="line">  cyclicBarrierSimple.await(); </span><br><span class="line">&#125;</span><br><span class="line">IntStream.range(<span class="number">0</span>,n).forEach(i -&gt; <span class="keyword">new</span> Thread(run).start());</span><br></pre></td></tr></table></figure>
<h2 id="V-并发容器"><a href="#V-并发容器" class="headerlink" title="V 并发容器"></a>V 并发容器</h2><h3 id="J-U-C-CopyOnWrite系列容器-vs-Collections-synchronizedCollection"><a href="#J-U-C-CopyOnWrite系列容器-vs-Collections-synchronizedCollection" class="headerlink" title="J.U.C.CopyOnWrite系列容器 vs. Collections.synchronizedCollection"></a>J.U.C.CopyOnWrite系列容器 vs. Collections.synchronizedCollection</h3><blockquote>
<ul>
<li><code>java.util.concurrent.CopyOnWriteArrayList</code> vs. <code>Collections.synchronizedList</code></li>
<li><code>java.util.concurrent.CopyOnWriteArraySet</code> vs. <code>Collections.synchronizedSet</code> </li>
</ul>
</blockquote>
<ul>
<li>CopyOnWrite 写得慢(每次都是复制), 读得快, 但读的是快照, 实时性差.</li>
<li>synchronizedCollection 说有操作均加锁, 相比于CopyOnWrite系列容器读得慢, 但写得快, 且实时性很强.</li>
</ul>
<h3 id="J-U-C-ConcurrentHashMap-vs-Hashtable-vs-Collections-synchronizedMap"><a href="#J-U-C-ConcurrentHashMap-vs-Hashtable-vs-Collections-synchronizedMap" class="headerlink" title="J.U.C.ConcurrentHashMap vs. Hashtable vs.  Collections.synchronizedMap"></a>J.U.C.ConcurrentHashMap vs. Hashtable vs.  Collections.synchronizedMap</h3><ul>
<li>ConcurrentHashMap: 将table分区, 对某些需要的区域进行加锁.</li>
<li>Hashtable: 将整个table全部加锁.</li>
<li>synchronizedMap: 对所有操作进行加锁.</li>
</ul>
<h3 id="阻塞队列-vs-并发队列-ArrayBlockingQueue-vs-ConcurrentLinkedQueue-vs-LinkedBlockingQueue"><a href="#阻塞队列-vs-并发队列-ArrayBlockingQueue-vs-ConcurrentLinkedQueue-vs-LinkedBlockingQueue" class="headerlink" title="阻塞队列 vs. 并发队列: ArrayBlockingQueue vs. ConcurrentLinkedQueue vs. LinkedBlockingQueue"></a>阻塞队列 vs. 并发队列: ArrayBlockingQueue vs. ConcurrentLinkedQueue vs. LinkedBlockingQueue</h3><blockquote>
<ul>
<li><code>java.util.concurrent.ArrayBlockingQueue</code></li>
<li><code>java.util.concurrent.ConcurrentLinkedQueue</code></li>
<li><code>java.util.concurrent.LinkedBlockingQueue</code></li>
</ul>
</blockquote>
<ul>
<li>ArrayBlockingQueue: 使用锁 + 数组.</li>
<li>ConcurrentLinkedQueue: 使用CAS + 链表.</li>
<li>LinkedBlockingQueue: 使用锁 + 链表</li>
</ul>
<blockquote>
<p> <strong>J.U.C.SynchronousQueue</strong>  —— 同步队列</p>
<p> 并不是一个实际可用于存储的队列, 它是同步队列, 用于同步生产者消费者.</p>
<p> <code>SynchronousQueue</code>本身无容量, 所有pull()进去的对象所在的线程都会阻塞, take()会恢复被阻塞的线程.</p>
<p> 类似于PV操作.</p>
</blockquote>
<h2 id="VI-程序编译"><a href="#VI-程序编译" class="headerlink" title="VI 程序编译"></a>VI 程序编译</h2><h3 id="1-前端-Java语法糖"><a href="#1-前端-Java语法糖" class="headerlink" title="1. 前端:  Java语法糖"></a>1. 前端:  Java语法糖</h3><h3 id="泛型系统"><a href="#泛型系统" class="headerlink" title="泛型系统"></a>泛型系统</h3><blockquote>
<p> Java泛型是语法糖, 生成代码时, 会类型擦除, 变成裸类型.</p>
</blockquote>
<h4 id="1-泛型理论-协变-逆变-不变"><a href="#1-泛型理论-协变-逆变-不变" class="headerlink" title="1. 泛型理论: 协变, 逆变, 不变"></a>1. 泛型理论: 协变, 逆变, 不变</h4><blockquote>
<p>定义: </p>
<ul>
<li>若 child <strong>extends</strong> parent, 且 f(child) 也是 f(parent)的派生, 称 f()为<strong>协变</strong>.</li>
<li>若 child <strong>extends</strong> parent, 而 f(parent) 是f(child) 的派生, 称f()为<strong>逆变</strong>.</li>
<li>若child <strong>extends</strong> parent, 而f(parent) 和f(child)无派生关系, 称为<strong>不变</strong>.</li>
</ul>
</blockquote>
<p>协变的核心原理是<strong>子类型可以隐式转化为父类型</strong>.</p>
<p>例如 <code>Child extends Parent</code>且<code>List&lt;Child&gt; extends List&lt;Parent&gt;</code> , 我们称<code>List&lt;&gt;</code>是协变的.</p>
<p>试想一下:</p>
<p><code>childList = [ch0, ch1, ch2, ...]</code> , 当<code>parentList = childList</code> 的时候, 其内部<code>[p0 = (parent)ch0, p1 = (parent)ch1, ...]</code> 很符合情理.</p>
<p>然而, Java泛型是基于类型擦除的泛型, 编译之后所有类型均擦除为裸类型, 所以Java的泛型系统是<strong>不变</strong>的.</p>
<p>具体的:</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line">List&lt;Child&gt; <span class="comment">//  类型擦除 --&gt; List</span></span><br><span class="line">List&lt;Parent&gt; <span class="comment">// 类型擦除 --&gt; List</span></span><br></pre></td></tr></table></figure>
<p>编译后均是<code>List</code> 这个裸类型, 因为同为<code>List</code>类, 其之间不存在<code>extends</code> 或者 <code>super</code> 关系, 所以是<strong>不变</strong>.</p>
<h4 id="2-泛型符号-和-T的区别"><a href="#2-泛型符号-和-T的区别" class="headerlink" title="2. 泛型符号: ? 和 T的区别"></a>2. 泛型符号: ? 和 T的区别</h4><p>T , K, V等字母: 在<strong>定义泛型类型</strong>的时候使用, 定义后在后续的代码块中使用.</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">//类</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">ClassA</span>&lt;<span class="title">T</span> <span class="keyword">extends</span> <span class="title">Comparable</span>&gt; </span>&#123;</span><br><span class="line">  <span class="keyword">private</span> T t;</span><br><span class="line">  <span class="function">T <span class="title">getT</span><span class="params">()</span> </span>&#123; ... &#125;</span><br><span class="line">  <span class="function"><span class="keyword">void</span> <span class="title">set</span><span class="params">(T t)</span></span>&#123; ... &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">//或者函数</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span>&lt;T extends Comparable&gt; <span class="function"><span class="keyword">void</span> <span class="title">test</span><span class="params">(T t)</span></span>&#123; ... &#125;</span><br></pre></td></tr></table></figure>
<p>? 通配符: 在<strong>使用泛型对象</strong>的时候使用, 限定泛型范围.</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">//例如, List&lt;T&gt; 是一个已经定义好的泛型类型, 是一个泛型容器.</span></span><br><span class="line"><span class="keyword">public</span> App&#123;</span><br><span class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">copy</span><span class="params">(List&lt;? <span class="keyword">super</span> Animal&gt; dst, List&lt;? extends Animal&gt; src)</span></span>&#123;</span><br><span class="line">    </span><br><span class="line">  &#125;</span><br><span class="line">  <span class="comment">//...</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>通配符指代:</p>
<p>若存在关系</p>
<p><img src="/images/Java_Note/mermaid-diagram-20200731122659.png" alt="类"></p>
<p>那么</p>
<ul>
<li><p><code>? extends Dog</code> 指代的类型是: Husky, Alaskan, Samoyed以及Dog自己.</p>
</li>
<li><p><code>? super Dog</code> 指代的类型是: Object, Animal, Wolf 以及Dog自己.</p>
</li>
</ul>
<p>用途:</p>
<ul>
<li><code>? extends Dog</code> 也称生产类型. 在数据拷贝的时候, 属于生产方.</li>
<li><code>? super Dog</code> 也称消费类型. 在数据拷贝的时候, 数据消耗方.</li>
</ul>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> App&#123;</span><br><span class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">copy</span><span class="params">(List&lt;? <span class="keyword">super</span> Dog&gt; dst, List&lt;? extends Dog&gt; src)</span></span>&#123;</span><br><span class="line">    <span class="comment">// 从子类拷贝数据到父类, 可以保证类型转换到安全.</span></span><br><span class="line">    src.forEach(dst::add);</span><br><span class="line">  &#125;</span><br><span class="line">  <span class="comment">//...</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3 id="2-后端-JIT-编译优化"><a href="#2-后端-JIT-编译优化" class="headerlink" title="2. 后端: JIT, 编译优化"></a>2. 后端: JIT, 编译优化</h3><p><strong>逃逸分析</strong></p>
<p>栈上分配: 无线程逃逸, 将堆中的对象放入栈中.</p>
<p>标量替换: 无方法逃逸, 将符合对象拆分成一个个基本类型.</p>
<h2 id="VII-Java-NIO"><a href="#VII-Java-NIO" class="headerlink" title="VII Java NIO"></a>VII Java NIO</h2><table>
<thead>
<tr>
<th>I/O模型</th>
<th>原理</th>
<th>缺点</th>
</tr>
</thead>
<tbody>
<tr>
<td>BlockingI/O</td>
<td>数据等待、数据传输时均被阻塞.</td>
<td>直接阻塞进程</td>
</tr>
<tr>
<td>non-Block I/O</td>
<td>用户进程循环询问Kernel; 请求后直接得到结果:数据或者错误.</td>
<td>轮询消耗CPU资源</td>
</tr>
<tr>
<td>多路复用</td>
<td>多个I/O注册到同一个复用器上, 若无可读的数据进程阻塞;  若有可用数据则唤醒进程进行读取.<br>被select函数阻塞, 而不是被Socket I/O阻塞.</td>
<td>实现难度大</td>
</tr>
<tr>
<td>AIO</td>
<td>利用系统调用epoll, 内核准备完成后通知进程, 实现真正的异步IO.</td>
<td>实现难度大</td>
</tr>
</tbody>
</table>
<h3 id="java-nio-ByteBuffer"><a href="#java-nio-ByteBuffer" class="headerlink" title="java.nio.ByteBuffer"></a>java.nio.ByteBuffer</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">flip(); <span class="comment">//保留limit到上次的位置, position归零, 用于读之前.</span></span><br><span class="line">clear(); <span class="comment">//全部归零, limit=capacity</span></span><br><span class="line">rewind();<span class="comment">//limit, position归零, 用于重写之前.</span></span><br></pre></td></tr></table></figure>
<h3 id="java-nio-channels"><a href="#java-nio-channels" class="headerlink" title="java.nio.channels.*"></a>java.nio.channels.*</h3><h2 id="杂项"><a href="#杂项" class="headerlink" title="杂项"></a>杂项</h2><h3 id="HashMap-原理"><a href="#HashMap-原理" class="headerlink" title="HashMap 原理"></a>HashMap 原理</h3><p><strong>存储原理</strong></p>
<ul>
<li>HashMap由一个数组组成, 使用拉链法进行存储.</li>
<li>当某个桶中节点大于等于8个, 链表化为红黑树.</li>
<li>当某个桶中节点小于等于6个时, 红黑树化为链表.</li>
</ul>
<blockquote>
<blockquote>
<p><strong>大于等于8个元素时, 为什么要化为红黑树?</strong></p>
<p>元素过多, 树形数据结构的搜索速度是$O(\lg n)$ 而线性结构是$O(n)$ .</p>
<p>所以应该使用BST之类的树, 例如AVL, 红黑树.</p>
</blockquote>
<blockquote>
<p><strong>为什么使用红黑树这个结构?</strong> </p>
<p>红黑树等价于4阶B树, 在失衡的时候, 其等价于B树2、3阶的节点吸收了失衡, 相比于AVL减少了旋转次数.</p>
</blockquote>
<blockquote>
<p><strong>为什么化为树的阈值是8</strong>? </p>
<ul>
<li>TreeNodes占用空间是普通Nodes的两倍, 过早的化为树, 空间代价过高;过晚的化为树, 搜索的时间代价过高.</li>
<li>理想情况下随机hashCode算法下所有桶中节点的分布频率会遵循<strong>泊松分布</strong>, 当某个节点大于8的时候, 其概率为100万分之6. <a href="https://www.javazhiyin.com/34651.html" target="_blank" rel="noopener">*注</a> </li>
</ul>
</blockquote>
<blockquote>
<p><strong>为什么化为链表的阈值是6?</strong></p>
<p>因为在8个节点左右反复插入删除, 会导致反复的链表和树的转化, 引起波动.</p>
</blockquote>
</blockquote>
<h3 id="反射的代价"><a href="#反射的代价" class="headerlink" title="反射的代价"></a>反射的代价</h3><ul>
<li>检查方法可见性</li>
<li>校验参数</li>
<li>JIT无法优化</li>
</ul>
<h4 id="将非try-with的对象封装成可try-with的对象"><a href="#将非try-with的对象封装成可try-with的对象" class="headerlink" title="将非try-with的对象封装成可try-with的对象"></a>将非try-with的对象封装成可try-with的对象</h4><ul>
<li>实现AutoCloseable接口</li>
</ul>
<h3 id="线程异常捕获-——-UncaughtExceptionHandler"><a href="#线程异常捕获-——-UncaughtExceptionHandler" class="headerlink" title="线程异常捕获 —— UncaughtExceptionHandler"></a>线程异常捕获 —— UncaughtExceptionHandler</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Thread.setDefaultUncaughtExceptionHandler((Thread t, Throwable e) -&gt;&#123;</span><br><span class="line">		System.err.println(t.getName() + <span class="string">" "</span> + e);</span><br><span class="line">&#125;);</span><br></pre></td></tr></table></figure>
<h3 id="写单例需要考虑的"><a href="#写单例需要考虑的" class="headerlink" title="写单例需要考虑的"></a>写单例需要考虑的</h3><ul>
<li>初始化的线程安全.</li>
<li>反射干涉内部成员的初始化.</li>
<li>序列化带来的问题.</li>
<li>效率.</li>
</ul>
<h4 id="解决线程安全-amp-反射-amp-序列化"><a href="#解决线程安全-amp-反射-amp-序列化" class="headerlink" title="解决线程安全 &amp; 反射 &amp; 序列化"></a>解决线程安全 &amp; 反射 &amp; 序列化</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">SignletonClass</span> <span class="keyword">implements</span> <span class="title">Serializable</span></span>&#123;</span><br><span class="line">  <span class="function"><span class="keyword">private</span> <span class="title">SignletonClass</span><span class="params">()</span></span>&#123;</span><br><span class="line">    <span class="keyword">if</span>(DataHolder.LAZY != <span class="keyword">null</span>) <span class="keyword">throw</span> <span class="keyword">new</span> RuntimeException(<span class="string">"Error"</span>); <span class="comment">//解决反射</span></span><br><span class="line">  &#125;</span><br><span class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> SignletonClass <span class="title">getInstance</span><span class="params">()</span></span>&#123;</span><br><span class="line">    <span class="keyword">return</span> InstanceHolder.LAZY;</span><br><span class="line">  &#125;</span><br><span class="line">  <span class="comment">//解决线程安全: 内部类一定会在getInstance()调用之前初始化, 避免了线程安全问题.</span></span><br><span class="line">  <span class="keyword">private</span> <span class="keyword">static</span> <span class="class"><span class="keyword">class</span> <span class="title">InstanceHolder</span></span>&#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> SignletonClass LAZY = <span class="keyword">new</span> SignletonClass(); </span><br><span class="line">  &#125;</span><br><span class="line">  </span><br><span class="line">  <span class="meta">@override</span></span><br><span class="line">  <span class="function"><span class="keyword">private</span> Object <span class="title">readResolve</span><span class="params">()</span></span>&#123; <span class="comment">//重写readResolve()解决序列化</span></span><br><span class="line">    <span class="keyword">return</span> SignletonClass.getInstance();</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h4 id="枚举式单例"><a href="#枚举式单例" class="headerlink" title="枚举式单例"></a>枚举式单例</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">enum</span> EnumSingleton&#123; <span class="comment">// 枚举类自带屏蔽反射, 序列化, 其初始化也是线程安全的, 效率也高.</span></span><br><span class="line">  INSTANCE;</span><br><span class="line">  <span class="keyword">private</span> Object data;</span><br><span class="line">  <span class="comment">//getter &amp; setter..</span></span><br><span class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> EnumSingleton <span class="title">getInstance</span><span class="params">()</span></span>&#123;</span><br><span class="line">    <span class="keyword">return</span> INSTANCE;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h4 id="容器单例模式"><a href="#容器单例模式" class="headerlink" title="容器单例模式"></a>容器单例模式</h4><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">ContainerSingleton</span> </span>&#123;</span><br><span class="line">  <span class="function"><span class="keyword">private</span> <span class="title">ContainerSingleton</span><span class="params">()</span></span>&#123;&#125;</span><br><span class="line">  <span class="keyword">private</span> <span class="keyword">static</span> Map&lt;String, Object&gt; ioc = <span class="keyword">new</span> ConcurrentHashMap&lt;&gt;();</span><br><span class="line">  <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> Object <span class="title">getBean</span><span class="params">(String className)</span></span>&#123;</span><br><span class="line">    <span class="keyword">synchronized</span>(ioc)&#123;</span><br><span class="line">      <span class="comment">//...</span></span><br><span class="line">    &#125;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<h3 id="String-vs-StringBuffer-vs-StringBuilder"><a href="#String-vs-StringBuffer-vs-StringBuilder" class="headerlink" title="String vs. StringBuffer vs. StringBuilder"></a>String vs. StringBuffer vs. StringBuilder</h3><ul>
<li>StringBuffer和StringBuilder在拼接字符串的时候均不会产生新的对象, 用法大体相同, 只是前者线程安全, 后者线程不安全.</li>
<li>StringBuilder的速度比StringBuffer高.</li>
</ul>
]]></content>
      <tags>
        <tag>java</tag>
      </tags>
  </entry>
</search>